32 citations · 187 across the 20 of their papers we have counts for
3 papers · 1 filter
On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks
Umut Şimşekli, Mert Gürbüzbalaban, Thanh Huy Nguyen +2
The gradient noise (GN) in the stochastic gradient descent (SGD) algorithm is often considered to be Gaussian in the large data regime by assuming that the \emph{classical} central…
First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise
Thanh Huy Nguyen, Umut Şimşekli, Mert Gürbüzbalaban +1
Stochastic gradient descent (SGD) has been widely used in machine learning due to its computational efficiency and favorable generalization properties. Recently, it has been empiri…
Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization
Thanh Huy Nguyen, Umut Şimşekli, Gaël Richard
Recent studies on diffusion-based sampling methods have shown that Langevin Monte Carlo (LMC) algorithms can be beneficial for non-convex optimization, and rigorous theoretical gua…