48 citations · 100 across the 9 of their papers we have counts for
Showing stat.MLShow all
2 papers · 1 filter
stat.ML2019★ 21 cited
On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks
Umut Şimşekli, Mert Gürbüzbalaban, Thanh Huy Nguyen +2
The gradient noise (GN) in the stochastic gradient descent (SGD) algorithm is often considered to be Gaussian in the large data regime by assuming that the \emph{classical} central…
stat.ML2018
Comparing Dynamics: Deep Neural Networks versus Glassy Systems
M. Baity-Jesi, L. Sagun, M. Geiger +6
We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are (…