Adaptive Weight Decay for Deep Neural Networks

Kensuke Nakamura, Byung-Woo Hong

Regularization in the optimization of deep neural networks is often critical to avoid undesirable over-fitting leading to better generalization of model. One of the most popular re…