Irreversible Langevin samplers and variance reduction: a large deviation approach

Abstract

In order to sample from a given target distribution (often of Gibbs type), the Monte Carlo Markov chain method consists in constructing an ergodic Markov process whose invariant measure is the target distribution. By sampling the Markov process one can then compute, approximately, expectations of observables with respect to the target distribution. Often the Markov processes used in practice are time-reversible (i.e., they satisfy detailed balance), but our main goal here is to assess and quantify how the addition of a non-reversible part to the process can be used to improve the sampling properties. We focus on the diffusion setting (overdamped Langevin equations) where the drift consists of a gradient vector field as well as another drift which breaks the reversibility of the process but is chosen to preserve the Gibbs measure. In this paper we use the large deviation rate function for the empirical measure as a tool to analyze the speed of convergence to the invariant measure. We show that the addition of an irreversible drift leads to a larger rate function and it strictly improves the speed of convergence of ergodic average for (generic smooth) observables. We also deduce from this result that the asymptotic variance decreases under the addition of the irreversible drift and we give an explicit characterization of the observables whose variance is not reduced reduced, in terms of a nonlinear Poisson equation. Our theoretical results are illustrated and supplemented by numerical simulations.

arXiv comment23 pages, 5 figures

References in corpus (4)

• General state space Markov chains and MCMC algorithms
• Optimal non-reversible linear drift for the convergence to equilibrium of a diffusion
• Accelerating diffusions
• An Infinite Swapping Approach to the Rare-Event Sampling Problem

Cited by in corpus (23)

• The Zig-Zag Process and Super-Efficient Sampling for Bayesian Analysis of Big Data
• Variance Reduction using Nonreversible Langevin Samplers
• Non-reversible Metropolis-Hastings
• Ergodicity of the zigzag process
• A piecewise deterministic scaling limit of Lifted Metropolis-Hastings in the Curie-Weiss model
• Improving the convergence of reversible samplers
• Acceleration of convergence to equilibrium in Markov chains by breaking detailed balance
• Limit theorems for the Zig-Zag process
• Sampling efficiency of transverse forces in dense liquids
• Using Perturbed Underdamped Langevin Dynamics to Efficiently Sample from Probability Distributions
• Role of current fluctuations in nonreversible samplers
• Bayesian Mechanics for Stationary Processes
• Large deviations for the Skew-Detailed-Balance Lifted-Markov processes to sample the equilibrium distribution of the Curie-Weiss model
• Irreversible Samplers from Jump and Continuous Markov Processes
• Asymmetric Thermal Relaxation in Driven Systems: Rotations go Opposite Ways
• Accelerating, to some extent, the $p$-spin dynamics
• Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents
• Irreversible Boltzmann samplers in dense liquids: weak-coupling approximation and mode-coupling theory
• Equilibrium large deviations for mean-field systems with translation invariance
• Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: a stochastic control approach
• On decompositions of non-reversible processes
• Generalizing Parallel Replica Dynamics: Trajectory Fragments, Asynchronous Computing, and PDMPs
• Description of a stochastic system by a nonadapted stochastic process