A theoretical comparison of the data augmentation, marginal augmentation and PX-DA algorithms

Abstract

The data augmentation (DA) algorithm is a widely used Markov chain Monte Carlo (MCMC) algorithm that is based on a Markov transition density of the form $p(x|x^{\prime })={\int }_{\mathsf{Y}}{f}_{X|Y}(x|y)f_{Y|X}(y|x^{\prime })dy$, where $f_{X|Y}$ and $f_{Y|X}$ are conditional densities. The PX-DA and marginal augmentation algorithms of Liu and Wu [J. Amer. Statist. Assoc. 94 (1999) 1264--1274] and Meng and van Dyk [Biometrika 86 (1999) 301--320] are alternatives to DA that often converge much faster and are only slightly more computationally demanding. The transition densities of these alternative algorithms can be written in the form $p_R(x|x^{\prime })={\int }_{\mathsf{Y}}{\int }_{\mathsf{Y}}{f}_{X|Y}(x|y^{\prime })R(y,d{y}^{\prime })f_{Y|X}(y|x^{\prime })dy$, where $R$ is a Markov transition function on $\mathsf{Y}$. We prove that when $R$ satisfies certain conditions, the MCMC algorithm driven by $p_R$ is at least as good as that driven by $p$ in terms of performance in the central limit theorem and in the operator norm sense. These results are brought to bear on a theoretical comparison of the DA, PX-DA and marginal augmentation algorithms. Our focus is on situations where the group structure exploited by Liu and Wu is available. We show that the PX-DA algorithm based on Haar measure is at least as good as any PX-DA algorithm constructed using a proper prior on the group.

Published in at http://dx.doi.org/10.1214/009053607000000569 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• General state space Markov chains and MCMC algorithms
• Variance bounding Markov chains

Cited by in corpus (13)

• A Short History of Markov Chain Monte Carlo: Subjective Recollections from Incomplete Data
• A spectral analytic comparison of trace-class data augmentation algorithms and their sandwich variants
• Estimation in Dirichlet random effects models
• Cross-Fertilizing Strategies for Better EM Mountain Climbing and DA Field Exploration: A Graphical Guide Book
• Improving the Convergence Properties of the Data Augmentation Algorithm with an Application to Bayesian Mixture Modeling
• Convergence properties of Gibbs samplers for Bayesian probit regression with proper priors
• Parameter Expansion and Efficient Inference
• Peskun-Tierney ordering for Markov chain and process Monte Carlo: beyond the reversible scenario
• Scalable Bayesian shrinkage and uncertainty quantification in high-dimensional regression
• Estimating the spectral gap of a trace-class Markov operator
• Convergence analysis of the block Gibbs sampler for Bayesian probit linear mixed models with improper priors
• Fast Markov chain Monte Carlo for high dimensional Bayesian regression models with shrinkage priors
• A Pólya-Gamma Sampler for a Generalized Logistic Regression