169 citations · 253 across the 12 of their papers we have counts for
4 papers · 1 filter
Stochastic Gradient Annealed Importance Sampling for Efficient Online Marginal Likelihood Estimation
Scott A. Cameron, Hans C. Eggers, Steve Kroon
We consider estimating the marginal likelihood in settings with independent and identically distributed (i.i.d.) data. We propose estimating the predictive distributions in a seque…
If dropout limits trainable depth, does critical initialisation still matter? A large-scale statistical analysis on ReLU networks
Arnu Pretorius, Elan van Biljon, Benjamin van Niekerk +6
Recent work in signal propagation theory has shown that dropout limits the depth to which information can propagate through a neural network. In this paper, we investigate the effe…
Critical initialisation for deep signal propagation in noisy rectifier neural networks
Arnu Pretorius, Elan Van Biljon, Steve Kroon +1
Stochastic regularisation is an important weapon in the arsenal of a deep learning practitioner. However, despite recent theoretical advances, our understanding of how noise influe…
Learning Dynamics of Linear Denoising Autoencoders
Arnu Pretorius, Steve Kroon, Herman Kamper
Denoising autoencoders (DAEs) have proven useful for unsupervised representation learning, but a thorough theoretical understanding is still lacking of how the input noise influenc…