Optimal scaling for partially updating MCMC algorithms

Abstract

In this paper we shall consider optimal scaling problems for high-dimensional Metropolis--Hastings algorithms where updates can be chosen to be lower dimensional than the target density itself. We find that the optimal scaling rule for the Metropolis algorithm, which tunes the overall algorithm acceptance rate to be 0.234, holds for the so-called Metropolis-within-Gibbs algorithm as well. Furthermore, the optimal efficiency obtainable is independent of the dimensionality of the update rule. This has important implications for the MCMC practitioner since high-dimensional updates are generally computationally more demanding, so that lower-dimensional updates are therefore to be preferred. Similar results with rather different conclusions are given for so-called Langevin updates. In this case, it is found that high-dimensional updates are frequently most efficient, even taking into account computing costs.

Published at http://dx.doi.org/10.1214/105051605000000791 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Cited by in corpus (20)

• A Survey of Monte Carlo Methods for Parameter Estimation
• Optimal scalings for local Metropolis--Hastings chains on nonproduct targets in high dimensions
• The Random Walk Metropolis: Linking Theory and Practice Through a Case Study
• Stochastic partial differential equation based modelling of large space-time data sets
• Weak convergence of Metropolis algorithms for non-i.i.d. target distributions
• Component-Wise Markov Chain Monte Carlo: Uniform and Geometric Ergodicity under Mixing and Composition
• Fast Markov chain Monte Carlo sampling for sparse Bayesian inference in high-dimensional inverse problems using L1-type priors
• Optimal scaling of random walk Metropolis algorithms with discontinuous target densities
• Quantifying uncertainty and dynamical changes in multi-species fishing mortality rates, catches and biomass by combining state-space and mechanistic multi-species models
• Weak Convergence and Optimal Tuning of the Reversible Jump Algorithm
• Complexity Bounds for MCMC via Diffusion Limits
• Optimal Scaling of Random-Walk Metropolis Algorithms on General Target Distributions
• Symmetry breaking at a topological phase transition
• MALA-within-Gibbs samplers for high-dimensional distributions with sparse conditional structure
• Emergent electrostatics in planar XY spin models: the bridge connecting topological order with broken $U(1)$ symmetry
• Optimal Scaling and Shaping of Random Walk Metropolis via Diffusion Limits of Block-I.I.D. Targets
• On Single Variable Transformation Approach to Markov Chain Monte Carlo
• Hidden Markov Models Applied To Intraday Momentum Trading With Side Information
• Optimal scaling of the independence sampler: Theory and Practice
• The two subset recurrent property of Markov chains