The Fundamental Incompatibility of Hamiltonian Monte Carlo and Data Subsampling

Abstract

Leveraging the coherent exploration of Hamiltonian flow, Hamiltonian Monte Carlo produces computationally efficient Monte Carlo estimators, even with respect to complex and high-dimensional target distributions. When confronted with data-intensive applications, however, the algorithm may be too expensive to implement, leaving us to consider the utility of approximations such as data subsampling. In this paper I demonstrate how data subsampling fundamentally compromises the efficient exploration of Hamiltonian flow and hence the scalable performance of Hamiltonian Monte Carlo itself.

arXiv comment8 pages, 5 figures

References in corpus (2)

• Optimizing The Integrator Step Size for Hamiltonian Monte Carlo
• (Non-) asymptotic properties of Stochastic Gradient Langevin Dynamics

Cited by in corpus (9)

• On Markov chain Monte Carlo methods for tall data
• On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators
• Coresets for Scalable Bayesian Logistic Regression
• What Are Bayesian Neural Network Posteriors Really Like?
• Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families
• Relativistic accretion disk reflection in AGN X-ray spectra at z=0.5--4: a study of four \textit{Chandra} deep fields
• PASS-GLM: polynomial approximate sufficient statistics for scalable Bayesian GLM inference
• Sparse Variational Inference: Bayesian Coresets from Scratch
• Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases