Adaptive Path Sampling in Metastable Posterior Distributions
arXiv:2009.00471
Abstract
The normalizing constant plays an important role in Bayesian computation, and there is a large literature on methods for computing or approximating normalizing constants that cannot be evaluated in closed form. When the normalizing constant varies by orders of magnitude, methods based on importance sampling can require many rounds of tuning. We present an improved approach using adaptive path sampling, iteratively reducing gaps between the base and target. Using this adaptive strategy, we develop two metastable sampling schemes. They are automated in Stan and require little tuning. For a multimodal posterior density, we equip simulated tempering with a continuous temperature. For a funnel-shaped entropic barrier, we adaptively increase mass in bottleneck regions to form an implicit divide-and-conquer. Both approaches empirically perform better than existing methods for sampling from metastable distributions, including higher accuracy and computation efficiency.
References in corpus (12)
- Statistically optimal analysis of samples from multiple equilibrium states
- Simulated Tempering: A New Monte Carlo Scheme
- A Conceptual Introduction to Hamiltonian Monte Carlo
- Sparsity information and regularization in the horseshoe and other shrinkage priors
- Pareto Smoothed Importance Sampling
- Conditions for rapid mixing of parallel and simulated tempering on multimodal distributions
- Implicitly Adaptive Importance Sampling
- Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors
- Kernel Adaptive Metropolis-Hastings
- Does Hamiltonian Monte Carlo mix faster than a random walk on multimodal densities?
- Unbiased estimation of log normalizing constants with applications to Bayesian cross-validation
- Adiabatic Monte Carlo