Weight Rescaling: Effective and Robust Regularization for Deep Neural Networks with Batch Normalization

Ziquan Liu, Yufei Cui, Jia Wan +2

Weight decay is often used to ensure good generalization in the training practice of deep neural networks with batch normalization (BN-DNNs), where some convolution layers are inva…