20 citations · 27 across the 3 of their papers we have counts for
Showing stat.MLShow all
2 papers · 1 filter
stat.ML2019
If dropout limits trainable depth, does critical initialisation still matter? A large-scale statistical analysis on ReLU networks
Arnu Pretorius, Elan van Biljon, Benjamin van Niekerk +6
Recent work in signal propagation theory has shown that dropout limits the depth to which information can propagate through a neural network. In this paper, we investigate the effe…
stat.ML2018
Critical initialisation for deep signal propagation in noisy rectifier neural networks
Arnu Pretorius, Elan Van Biljon, Steve Kroon +1
Stochastic regularisation is an important weapon in the arsenal of a deep learning practitioner. However, despite recent theoretical advances, our understanding of how noise influe…