Auxiliary-variable Exact Hamiltonian Monte Carlo Samplers for Binary Distributions

Abstract

We present a new approach to sample from generic binary distributions, based on an exact Hamiltonian Monte Carlo algorithm applied to a piecewise continuous augmentation of the binary distribution of interest. An extension of this idea to distributions over mixtures of binary and possibly-truncated Gaussian or exponential variables allows us to sample from posteriors of linear and probit regression models with spike-and-slab priors and truncated parameters. We illustrate the advantages of these algorithms in several examples in which they outperform the Metropolis or Gibbs samplers.

11 pages, 4 figures. Proceedings of the 27th Annual Conference Neural Information Processing Systems (NIPS), 2013

References in corpus (3)

• The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
• Spike-and-Slab Sparse Coding for Unsupervised Feature Discovery
• Bayesian and L1 Approaches to Sparse Unsupervised Learning

Cited by in corpus (11)

• Discontinuous Hamiltonian Monte Carlo for discrete parameters and discontinuous likelihoods
• Modified Hamiltonian Monte Carlo for Bayesian inference
• Conditional independence by typing
• Bayesian spike inference from calcium imaging data
• Accurate and Efficient Estimation of Small P-values with the Cross-Entropy Method: Applications in Genomic Data Analysis
• The Ising Model with Hybrid Monte Carlo
• Roll-back Hamiltonian Monte Carlo
• Binary Bouncy Particle Sampler
• Recycling intermediate steps to improve Hamiltonian Monte Carlo
• Bayesian Inference with the l1-ball Prior: Solving Combinatorial Problems with Exact Zeros
• Sampling in Combinatorial Spaces with SurVAE Flow Augmented MCMC