A Complete Recipe for Stochastic Gradient MCMC

Abstract

Many recent Markov chain Monte Carlo (MCMC) samplers leverage continuous dynamics to define a transition kernel that efficiently explores a target distribution. In tandem, a focus has been on devising scalable variants that subsample the data and use stochastic gradients in place of full-data gradients in the dynamic simulations. However, such stochastic gradient MCMC samplers have lagged behind their full-data counterparts in terms of the complexity of dynamics considered since proving convergence in the presence of the stochastic gradient noise is non-trivial. Even with simple dynamics, significant physical intuition is often required to modify the dynamical system to account for the stochastic gradient noise. In this paper, we provide a general recipe for constructing MCMC samplers--including stochastic gradient versions--based on continuous Markov processes specified via two matrices. We constructively prove that the framework is complete. That is, any continuous Markov process that provides samples from the target distribution can be written in our framework. We show how previous continuous-dynamic samplers can be trivially "reinvented" in our framework, avoiding the complicated sampler-specific proofs. We likewise use our recipe to straightforwardly propose a new state-adaptive sampler: stochastic gradient Riemann Hamiltonian Monte Carlo (SGRHMC). Our experiments on simulated data and a streaming Wikipedia analysis demonstrate that the proposed SGRHMC sampler inherits the benefits of Riemann HMC, with the scalability of stochastic gradient methods.

Cited by in corpus (55)

• The free energy principle made simpler but not too simple
• On the Random Batch Method for second order interacting particle systems
• Variance Reduction using Nonreversible Langevin Samplers
• Stochastic Gradient Descent as Approximate Bayesian Inference
• Entropy-SGD: Biasing Gradient Descent Into Wide Valleys
• What Are Bayesian Neural Network Posteriors Really Like?
• Underdamped Langevin MCMC: A non-asymptotic analysis
• Implicit Weight Uncertainty in Neural Networks
• A Variational Analysis of Stochastic Gradient Algorithms
• Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC
• Stochastic Quasi-Newton Langevin Monte Carlo
• Multiplicative noise and heavy tails in stochastic optimization
• Probabilistic Permutation Synchronization using the Riemannian Structure of the Birkhoff Polytope
• AutoLoss: Learning Discrete Schedules for Alternate Optimization
• On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo
• Bayesian Mechanics for Stationary Processes
• TherML: Thermodynamics of Machine Learning
• Riemannian Stein Variational Gradient Descent for Bayesian Inference
• Meta-Learning for Stochastic Gradient MCMC
• Stochastic Gradient MCMC Methods for Hidden Markov Models
• Confidence-Aware Learning for Camouflaged Object Detection
• Nonreversible Langevin Samplers: Splitting Schemes, Analysis and Implementation
• Consistency of Bayesian inference with Gaussian process priors for a parabolic inverse problem
• Bayesian Learning of Parameterised Quantum Circuits
• Softplus Regressions and Convex Polytopes
• Fast Simulation of Hyperplane-Truncated Multivariate Normal Distributions
• Bayesian Sparse learning with preconditioned stochastic gradient MCMC and its applications
• Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents
• A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributions
• A Convergence Analysis for A Class of Practical Variance-Reduction Stochastic Gradient MCMC
• Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Server
• Protecting Classifiers From Attacks
• Stochastic Gradient Monomial Gamma Sampler
• On Thompson Sampling with Langevin Algorithms
• On Stein Variational Neural Network Ensembles
• Gibbs Sampling the Posterior of Neural Networks
• An adaptive Hessian approximated stochastic gradient MCMC method
• Estimate exponential memory decay in Hidden Markov Model and its applications
• CaMKII activation supports reward-based neural network optimization through Hamiltonian sampling
• Collaborative Filtering with Side Information: a Gaussian Process Perspective
• Stochastic Particle-Optimization Sampling and the Non-Asymptotic Convergence Theory
• Energy-Based Models with Applications to Speech and Language Processing
• JaxSGMC: Modular stochastic gradient MCMC in JAX
• Stochastic Gradient Langevin Dynamics Algorithms with Adaptive Drifts
• On Connecting Stochastic Gradient MCMC and Differential Privacy
• High-Order Stochastic Gradient Thermostats for Bayesian Learning of Deep Models
• A Langevinized Ensemble Kalman Filter for Large-Scale Static and Dynamic Learning
• A Mathematical Walkthrough and Discussion of the Free Energy Principle
• Targeted stochastic gradient Markov chain Monte Carlo for hidden Markov models with rare latent states
• Isotropic SGD: a Practical Approach to Bayesian Posterior Sampling
• Stochastic Gradient Annealed Importance Sampling for Efficient Online Marginal Likelihood Estimation
• Stochastic Gradient MCMC with Stale Gradients
• The Dependent Random Measures with Independent Increments in Mixture Models
• Simulating the stochastic dynamics and cascade failure of power networks
• Asynchronous Stochastic Gradient MCMC with Elastic Coupling