Towards Unifying Hamiltonian Monte Carlo and Slice Sampling

Abstract

We unify slice sampling and Hamiltonian Monte Carlo (HMC) sampling, demonstrating their connection via the Hamiltonian-Jacobi equation from Hamiltonian mechanics. This insight enables extension of HMC and slice sampling to a broader family of samplers, called Monomial Gamma Samplers (MGS). We provide a theoretical analysis of the mixing performance of such samplers, proving that in the limit of a single parameter, the MGS draws decorrelated samples from the desired target distribution. We further show that as this parameter tends toward this limit, performance gains are achieved at a cost of increasing numerical difficulty and some practical convergence issues. Our theoretical results are validated with synthetic data and real-world applications.

updated version

Cited by in corpus (5)

• Discontinuous Hamiltonian Monte Carlo for discrete parameters and discontinuous likelihoods
• Modified Hamiltonian Monte Carlo for Bayesian inference
• Recycling intermediate steps to improve Hamiltonian Monte Carlo
• Understanding MCMC Dynamics as Flows on the Wasserstein Space
• Variational Langevin Hamiltonian Monte Carlo for Distant Multi-modal Sampling