32 citations · 187 across the 20 of their papers we have counts for
Showing 2019 · stat.MLShow all
2 papers · 2 filters
stat.ML2019★ 21 cited
On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks
Umut Şimşekli, Mert Gürbüzbalaban, Thanh Huy Nguyen +2
The gradient noise (GN) in the stochastic gradient descent (SGD) algorithm is often considered to be Gaussian in the large data regime by assuming that the \emph{classical} central…
stat.ML2019★ 11 cited
First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise
Thanh Huy Nguyen, Umut Şimşekli, Mert Gürbüzbalaban +1
Stochastic gradient descent (SGD) has been widely used in machine learning due to its computational efficiency and favorable generalization properties. Recently, it has been empiri…