A spectral gap for Metropolis-adjusted Langevin algorithm with a uniformly randomized step size

Abstract

Let $\pi (\mathrm{d}x)\propto e^{-U(x)}\mathrm{d}x$ on ${\mathbb{R}}^d$, where $U$ is continuously differentiable and $m$-strongly convex with a globally $L$-Lipschitz gradient, $0<m\le L<\infty$, and $\kappa =L/m$. Fixed-step Metropolis-adjusted Langevin algorithm (MALA) has known warm-start mixing-time upper bounds of order $\kappa \sqrt{d}$, up to logarithmic factors. A spectral-gap lower bound at the corresponding scale $(\kappa \sqrt{d}{)}^{-1}$ would give an upper bound of order $\kappa \sqrt{d}$ on the Monte Carlo asymptotic variance relative to independent sampling, uniformly over all square-integrable functions. However, when $\kappa$ is bounded away from~1, the best fixed-step spectral gap that can be guaranteed uniformly over this target class is at most of order $\max \{\log (\kappa d)/(\kappa d),e^{-cd}\}$ for some positive universal constant~$c$. We show that uniform randomization of the step size improves this worst-case guarantee. At each iteration, the algorithm draws $h\sim \mathrm{Unif}(0,H)$ and performs one ordinary MALA transition. Choosing $H$ of order $[L\sqrt{d(1+\log d+\log \kappa )}{]}^{-1}$ yields a right spectral-gap lower bound of order \[ \frac{1}{κ\sqrt{d(1+\log d+\logκ)}}, \] uniformly over the target class. Thus, given $\kappa >1$, for all square-integrable functions, the ratio of the Monte Carlo asymptotic variance relative to independence sampling has an upper bound of order $\sqrt{d\log d}$. This contrasts with fixed-MALA, where the ratio can be as bad as $d/\log d$ in thew worst-case scenario. The spectral gap also gives geometric convergence of the lazy kernel from every initial density in $L^2(\pi )$, central limit theorems, and nonstationary mean-square error bounds. This work was developed with substantial assistance from ChatGPT.