20 citations · 35 across the 7 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
stat.ML2020
A Random Matrix Theory Approach to Damping in Deep Learning
Diego Granziol, Nicholas Baskerville
We conjecture that the inherent difference in generalisation between adaptive and non-adaptive gradient methods in deep learning stems from the increased estimation noise in the fl…
math.PR2020★ 20 cited
The Loss Surfaces of Neural Networks with General Activation Functions
Nicholas P. Baskerville, Jonathan P. Keating, Francesco Mezzadri +1
The loss surfaces of deep neural networks have been the subject of several studies, theoretical and experimental, over the last few years. One strand of work considers the complexi…