20 citations · 26 across the 5 of their papers we have counts for
4 papers · 1 filter
Robust and Scalable SDE Learning: A Functional Perspective
Scott Cameron, Tyron Cameron, Arnu Pretorius +1
Stochastic differential equations provide a rich class of flexible generative models, capable of describing a wide range of spatio-temporal processes. A host of recent work looks t…
If dropout limits trainable depth, does critical initialisation still matter? A large-scale statistical analysis on ReLU networks
Arnu Pretorius, Elan van Biljon, Benjamin van Niekerk +6
Recent work in signal propagation theory has shown that dropout limits the depth to which information can propagate through a neural network. In this paper, we investigate the effe…
Critical initialisation for deep signal propagation in noisy rectifier neural networks
Arnu Pretorius, Elan Van Biljon, Steve Kroon +1
Stochastic regularisation is an important weapon in the arsenal of a deep learning practitioner. However, despite recent theoretical advances, our understanding of how noise influe…
Learning Dynamics of Linear Denoising Autoencoders
Arnu Pretorius, Steve Kroon, Herman Kamper
Denoising autoencoders (DAEs) have proven useful for unsupervised representation learning, but a thorough theoretical understanding is still lacking of how the input noise influenc…