Ergodicity of Approximate MCMC Chains with Applications to Large Data Sets
arXiv:1405.0182
Abstract
In many modern applications, difficulty in evaluating the posterior density makes performing even a single MCMC step slow. This difficulty can be caused by intractable likelihood functions, but also appears for routine problems with large data sets. Many researchers have responded by running approximate versions of MCMC algorithms. In this note, we develop quantitative bounds for showing the ergodicity of these approximate samplers. We then use these bounds to study the bias-variance trade-off of approximate MCMC algorithms. We apply our results to simple versions of recently proposed algorithms, including a variant of the "austerity" framework of Korratikara et al.
Substantially revised and shortened from the previous version
References in corpus (9)
- Stochastic Gradient Hamiltonian Monte Carlo
- Speeding Up MCMC by Efficient Data Subsampling
- On Markov chain Monte Carlo methods for tall data
- Big Learning with Bayesian Methods
- Noisy Monte Carlo: Convergence of Markov chains with approximate transition kernels
- Light and Widely Applicable MCMC: Approximate Bayesian Inference for Large Datasets
- Sublinear-Time Approximate MCMC Transitions for Probabilistic Programs
- Subgaussian concentration inequalities for geometrically ergodic Markov chains
- Analyzing statistical and computational tradeoffs of estimation procedures
Cited by in corpus (14)
- On Markov chain Monte Carlo methods for tall data
- Perturbation Bounds for Monte Carlo within Metropolis via Restricted Approximations
- Bayes Shrinkage at GWAS scale: Convergence and Approximation Theory of a Scalable MCMC Algorithm for the Horseshoe Prior
- Bayesian computation: a perspective on the current state, and sampling backwards and forwards
- Approximations of Geometrically Ergodic Reversible Markov Chains
- Big Learning with Bayesian Methods
- Explicit contraction rates for a class of degenerate and infinite-dimensional diffusions
- Flexible Bayesian Nonlinear Model Configuration
- Deep Bayesian regression models
- Sublinear-Time Approximate MCMC Transitions for Probabilistic Programs
- Light and Widely Applicable MCMC: Approximate Bayesian Inference for Large Datasets
- A fast asynchronous MCMC sampler for sparse Bayesian inference
- Sequential sampling of Gaussian process latent variable models
- Exploiting Multi-Core Architectures for Reduced-Variance Estimation with Intractable Likelihoods