Computing Bayes: From Then 'Til Now'
arXiv:2208.00646 · doi:10.1214/22-STS876
Abstract
This paper takes the reader on a journey through the history of Bayesian computation, from the 18th century to the present day. Beginning with the one-dimensional integral first confronted by Bayes in 1763, we highlight the key contributions of: Laplace, Metropolis (and, importantly, his co-authors!), Hammersley and Handscomb, and Hastings, all of which set the foundations for the computational revolution in the late 20th century -- led, primarily, by Markov chain Monte Carlo (MCMC) algorithms. A very short outline of 21st century computational methods -- including pseudo-marginal MCMC, Hamiltonian Monte Carlo, sequential Monte Carlo, and the various `approximate' methods -- completes the paper.
arXiv commentMaterial that appeared in an earlier paper, `Computing Bayes: Bayesian Computation from 1763 to the 21st Century' (arXiv:2004.06425) has been broken up into two separate papers: this historical overview of, and timeline for, all computational developments is retained; and a secondary paper (arXiv:2112.10342), which provides a more detailed review of 21st century
References in corpus (32)
- Variational Inference: A Review for Statisticians
- MCMC using Hamiltonian dynamics
- Simulated Tempering: A New Monte Carlo Scheme
- The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
- The pseudo-marginal approach for efficient Monte Carlo computations
- Adaptive approximate Bayesian computation
- A Conceptual Introduction to Hamiltonian Monte Carlo
- A Short History of Markov Chain Monte Carlo: Subjective Recollections from Incomplete Data
- Speeding Up MCMC by Efficient Data Subsampling
- The Zig-Zag Process and Super-Efficient Sampling for Bayesian Analysis of Big Data
- On Markov chain Monte Carlo methods for tall data
- Three discussions of the paper "sequential quasi-Monte Carlo sampling", by M. Gerber and N. Chopin
- On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods
- A Review of Multiple Try MCMC algorithms for Signal Processing
- Population Monte Carlo algorithms
- Piecewise Deterministic Markov Processes for Continuous-Time Monte Carlo
- Importance Tempering
- Delayed acceptance particle MCMC for exact inference in stochastic kinetic models
- Multiprocess parallel antithetic coupling for backward and forward Markov Chain Monte Carlo
- Piecewise Deterministic Markov Processes for Scalable Monte Carlo on Restricted Domains
- Speeding Up MCMC by Delayed Acceptance and Data Subsampling
- Consistency of Markov chain quasi-Monte Carlo on continuous state spaces
- Scalable Importance Tempering and Bayesian Variable Selection
- MCMC Using Ensembles of States for Problems with Fast and Slow Variables such as Gaussian Process Regression
- A vanilla Rao--Blackwellization of Metropolis--Hastings algorithms
- Approximating Bayes in the 21st Century
- Integrated Nested Laplace Approximations (INLA)
- Generalized Posteriors in Approximate Bayesian Computation
- Accelerating delayed-acceptance Markov chain Monte Carlo algorithms
- Variational Bayes under Model Misspecification
- A Common Derivation for Markov Chain Monte Carlo Algorithms with Tractable and Intractable Targets
- An attempt to trace the birth of importance sampling