Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion

Alexandru Meterez, Amir Joudaki, Francesco Orabona +3

Normalization layers are one of the key building blocks for deep neural networks. Several theoretical studies have shown that batch normalization improves the signal propagation, b…