Batch Normalization Preconditioning for Neural Network Training

Susanna Lange, Kyle Helfrich, Qiang Ye

Batch normalization (BN) is a popular and ubiquitous method in deep learning that has been shown to decrease training time and improve generalization performance of neural networks…