Regularization in ResNet with Stochastic Depth

Soufiane Hayou, Fadhel Ayed

Regularization plays a major role in modern deep learning. From classic techniques such as L1,L2 penalties to other noise-based methods such as Dropout, regularization often yields…