22 citations · 29 across the 6 of their papers we have counts for
6 papers
Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD than Constant Stepsize
Mert Gürbüzbalaban, Yuanhan Hu, Umut Şimşekli +1
Cyclic and randomized stepsizes are widely used in the deep learning practice and can often outperform standard stepsize choices such as constant stepsize in SGD. Despite their emp…
Penalized Overdamped and Underdamped Langevin Monte Carlo Algorithms for Constrained Sampling
Mert Gürbüzbalaban, Yuanhan Hu, Lingjiong Zhu
We consider the constrained sampling problem where the goal is to sample from a target distribution when is constrained to lie on a convex body $\mathca…
Heavy-Tail Phenomenon in Decentralized SGD
Mert Gurbuzbalaban, Yuanhan Hu, Umut Simsekli +2
Recent theoretical studies have shown that heavy-tails can emerge in stochastic optimization due to `multiplicative noise', even under surprisingly simple settings, such as linear…
Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
Mert Gürbüzbalaban, Xuefeng Gao, Yuanhan Hu +1
Stochastic gradient Langevin dynamics (SGLD) and stochastic gradient Hamiltonian Monte Carlo (SGHMC) are two popular Markov Chain Monte Carlo (MCMC) algorithms for Bayesian inferen…
Non-Convex Optimization via Non-Reversible Stochastic Gradient Langevin Dynamics
Yuanhan Hu, Xiaoyu Wang, Xuefeng Gao +2
Stochastic Gradient Langevin Dynamics (SGLD) is a powerful algorithm for optimizing a non-convex objective, where a controlled and properly scaled Gaussian noise is added to the st…
Fractional moment-preserving initialization schemes for training deep neural networks
Mert Gurbuzbalaban, Yuanhan Hu
A traditional approach to initialization in deep neural networks (DNNs) is to sample the network weights randomly for preserving the variance of pre-activations. On the other hand,…