Scalable Importance Tempering and Bayesian Variable Selection

Abstract

We propose a Monte Carlo algorithm to sample from high dimensional probability distributions that combines Markov chain Monte Carlo and importance sampling. We provide a careful theoretical analysis, including guarantees on robustness to high dimensionality, explicit comparison with standard Markov chain Monte Carlo methods and illustrations of the potential improvements in efficiency. Simple and concrete intuition is provided for when the novel scheme is expected to outperform standard schemes. When applied to Bayesian variable-selection problems, the novel algorithm is orders of magnitude more efficient than available alternative sampling schemes and enables fast and reliable fully Bayesian inferences with tens of thousand regressors.

arXiv commentOnline supplement not included

References in corpus (4)

• Simulated Tempering: A New Monte Carlo Scheme
• Random lasso
• On the Computational Complexity of MCMC-based Estimators in Large Samples
• Importance Tempering

Cited by in corpus (8)

• Projective Inference in High-dimensional Problems: Prediction and Feature Selection
• Linear Complexity Gibbs Sampling for Generalized Labeled Multi-Bernoulli Filtering
• Robust leave-one-out cross-validation for high-dimensional Bayesian models
• Computing Bayes: From Then 'Til Now'
• Sticky PDMP samplers for sparse and local inference problems
• A Metropolized adaptive subspace algorithm for high-dimensional Bayesian variable selection
• Approximate Laplace approximations for scalable model selection
• UbiQVision: Quantifying Uncertainty in XAI for Image Recognition