The promises and pitfalls of Stochastic Gradient Langevin Dynamics

Abstract

Stochastic Gradient Langevin Dynamics (SGLD) has emerged as a key MCMC algorithm for Bayesian learning from large scale datasets. While SGLD with decreasing step sizes converges weakly to the posterior distribution, the algorithm is often used with a constant step size in practice and has demonstrated successes in machine learning tasks. The current practice is to set the step size inversely proportional to $N$ where $N$ is the number of training samples. As $N$ becomes large, we show that the SGLD algorithm has an invariant probability measure which significantly departs from the target posterior and behaves like Stochastic Gradient Descent (SGD). This difference is inherently due to the high variance of the stochastic gradients. Several strategies have been suggested to reduce this effect; among them, SGLD Fixed Point (SGLDFP) uses carefully designed control variates to reduce the variance of the stochastic gradients. We show that SGLDFP gives approximate samples from the posterior distribution, with an accuracy comparable to the Langevin Monte Carlo (LMC) algorithm for a computational cost sublinear in the number of data points. We provide a detailed analysis of the Wasserstein distances between LMC, SGLD, SGLDFP and SGD and explicit expressions of the means and covariance matrices of their invariant distributions. Our findings are supported by limited numerical experiments.

Cited by in corpus (14)

• Analysis of Stochastic Gradient Descent in Continuous Time
• On the Convergence of Langevin Monte Carlo: The Interplay between Tail Growth and Smoothness
• On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation
• Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimization
• Hamiltonian Monte Carlo with Energy Conserving Subsampling
• Nonasymptotic analysis of Stochastic Gradient Hamiltonian Monte Carlo under local conditions for nonconvex optimization
• Efficient MCMC Sampling with Dimension-Free Convergence Rate using ADMM-type Splitting
• Is SGD a Bayesian sampler? Well, almost
• Mixing it up: A general framework for Markovian statistics
• Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets
• QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning
• Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks
• Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation function
• Variance reduction for dependent sequences with applications to Stochastic Gradient MCMC