Critical initialisation for deep signal propagation in noisy rectifier neural networks

Arnu Pretorius, Elan Van Biljon, Steve Kroon +1

Stochastic regularisation is an important weapon in the arsenal of a deep learning practitioner. However, despite recent theoretical advances, our understanding of how noise influe…