21 citations · 61 across the 8 of their papers we have counts for
3 papers · 1 filter
On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks
Umut Şimşekli, Mert Gürbüzbalaban, Thanh Huy Nguyen +2
The gradient noise (GN) in the stochastic gradient descent (SGD) algorithm is often considered to be Gaussian in the large data regime by assuming that the \emph{classical} central…
First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise
Thanh Huy Nguyen, Umut Şimşekli, Mert Gürbüzbalaban +1
Stochastic gradient descent (SGD) has been widely used in machine learning due to its computational efficiency and favorable generalization properties. Recently, it has been empiri…
Asynchronous Stochastic Quasi-Newton MCMC for Non-Convex Optimization
Umut Şimşekli, Çağatay Yıldız, Thanh Huy Nguyen +2
Recent studies have illustrated that stochastic gradient Markov Chain Monte Carlo techniques have a strong potential in non-convex optimization, where local and global convergence…