Sampling Constrained Continuous Probability Distributions: A Review

Abstract

The problem of sampling constrained continuous distributions has frequently appeared in many machine/statistical learning models. Many Monte Carlo Markov Chain (MCMC) sampling methods have been adapted to handle different types of constraints on the random variables. Among these methods, Hamilton Monte Carlo (HMC) and the related approaches have shown significant advantages in terms of computational efficiency compared to other counterparts. In this article, we first review HMC and some extended sampling methods, and then we concretely explain three constrained HMC-based sampling methods, reflection, reformulation, and spherical HMC. For illustration, we apply these methods to solve three well-known constrained sampling problems, truncated multivariate normal distributions, Bayesian regularized regression, and nonparametric density estimation. In this review, we also connect constrained sampling with another similar problem in the statistical design of experiments of constrained design space.

References in corpus (10)

• Optimal scaling of the random walk Metropolis on elliptically symmetric unimodal targets
• Optimal scaling of random walk Metropolis algorithms with discontinuous target densities
• Sampling with Riemannian Hamiltonian Monte Carlo in a Constrained Space
• Integrals over Gaussians under Linear Domain Constraints
• Efficient constrained sampling via the mirror-Langevin algorithm
• Bayesian Auxiliary Variable Model for Birth Records Data with Qualitative and Quantitative Responses
• Constrained Minimum Energy Designs
• Accelerating Bayesian inference of dependency between complex biological traits
• Rejection sampling from shape-constrained distributions in sublinear time
• Zigzag path connects two Monte Carlo samplers: Hamiltonian counterpart to a piecewise deterministic Markov process