Accelerated dimension-independent adaptive Metropolis
arXiv:1506.05741 · doi:10.1137/15M1026432
Abstract
This work considers black-box Bayesian inference over high-dimensional parameter spaces. The well-known adaptive Metropolis (AM) algorithm of (Haario etal. 2001) is extended herein to scale asymptotically uniformly with respect to the underlying parameter dimension for Gaussian targets, by respecting the variance of the target. The resulting algorithm, referred to as the dimension-independent adaptive Metropolis (DIAM) algorithm, also shows improved performance with respect to adaptive Metropolis on non-Gaussian targets. This algorithm is further improved, and the possibility of probing high-dimensional targets is enabled, via GPU-accelerated numerical libraries and periodically synchronized concurrent chains (justified a posteriori). Asymptotically in dimension, this GPU implementation exhibits a factor of four improvement versus a competitive CPU-based Intel MKL parallel version alone. Strong scaling to concurrent chains is exhibited, through a combination of longer time per sample batch (weak scaling) and yet fewer necessary samples to convergence. The algorithm performance is illustrated on several Gaussian and non-Gaussian target examples, in which the dimension may be in excess of one thousand.
References in corpus (13)
- Exchange Monte Carlo Method and Application to Spin Glass Simulations
- On the ergodicity properties of some adaptive MCMC algorithms
- On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods
- Dimension-independent likelihood-informed MCMC
- Likelihood-informed dimension reduction for nonlinear inverse problems
- Optimal scalings for local Metropolis--Hastings chains on nonproduct targets in high dimensions
- Diffusion limits of the random walk Metropolis algorithm in high dimensions
- Convergence of adaptive and interacting Markov chain Monte Carlo algorithms
- Optimal scaling and diffusion limits for the Langevin algorithm in high dimensions
- On the ergodicity of the adaptive Metropolis algorithm on unbounded domains
- Multiprocess parallel antithetic coupling for backward and forward Markov Chain Monte Carlo
- Proposals which speed-up function-space MCMC
- Solving Large-Scale PDE-constrained Bayesian Inverse Problems with Riemann Manifold Hamiltonian Monte Carlo