Optimizing The Integrator Step Size for Hamiltonian Monte Carlo

Abstract

Hamiltonian Monte Carlo can provide powerful inference in complex statistical problems, but ultimately its performance is sensitive to various tuning parameters. In this paper we use the underlying geometry of Hamiltonian Monte Carlo to construct a universal optimization criteria for tuning the step size of the symplectic integrator crucial to any implementation of the algorithm as well as diagnostics to monitor for any signs of invalidity. An immediate outcome of this result is that the suggested target average acceptance probability of 0.651 can be relaxed to $0.6\lesssim a\lesssim 0.9$ with larger values more robust in practice.

36 pages, 5 figures

References in corpus (2)

• The Geometric Foundations of Hamiltonian Monte Carlo
• Generalizing the No-U-Turn Sampler to Riemannian Manifolds

Cited by in corpus (22)

• A Survey of Stochastic Simulation and Optimization Methods in Signal Processing
• On the Geometric Ergodicity of Hamiltonian Monte Carlo
• Langevin and Hamiltonian based Sequential MCMC for Efficient Bayesian Filtering in High-dimensional Spaces
• Discontinuous Hamiltonian Monte Carlo for discrete parameters and discontinuous likelihoods
• Diagnosing Suboptimal Cotangent Disintegrations in Hamiltonian Monte Carlo
• Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration
• Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families
• The Fundamental Incompatibility of Hamiltonian Monte Carlo and Data Subsampling
• Fast mixing of Metropolized Hamiltonian Monte Carlo: Benefits of multi-step gradients
• Bayesian computation: a perspective on the current state, and sampling backwards and forwards
• K-CLASH: Strangulation and Ram Pressure Stripping in Galaxy Cluster Members at 0.3 < z < 0.6
• Hamiltonian Monte Carlo using an adjoint-differentiated Laplace approximation: Bayesian inference for latent Gaussian models and beyond
• A scalable optimal-transport based local particle filter
• DAmodel: Hierarchical Bayesian Modelling of DA White Dwarfs for Spectrophotometric Calibration
• A Condition Number for Hamiltonian Monte Carlo
• Focusing on Difficult Directions for Learning HMC Trajectory Lengths
• Markov Chain Monte Carlo Methods for Estimating Systemic Risk Allocations
• An Introduction to Hamiltonian Monte Carlo Method for Sampling
• Measure Transport with Kernel Stein Discrepancy
• A Bayesian Approach to Spherical Factor Analysis for Binary Data
• FlowSN: Neural Simulation-Based Inference under Realistic Selection Effects applied to Supernova Cosmology
• FunMC: A functional API for building Markov Chains