An approach to large-scale Quasi-Bayesian inference with spike-and-slab priors

Abstract

We propose a general framework using spike-and-slab prior distributions to aid with the development of high-dimensional Bayesian inference. Our framework allows inference with a general quasi-likelihood function. We show that highly efficient and scalable Markov Chain Monte Carlo (MCMC) algorithms can be easily constructed to sample from the resulting quasi-posterior distributions. We study the large scale behavior of the resulting quasi-posterior distributions as the dimension of the parameter space grows, and we establish several convergence results. In large-scale applications where computational speed is important, variational approximation methods are often used to approximate posterior distributions. We show that the contraction behaviors of the quasi-posterior distributions can be exploited to provide theoretical guarantees for their variational approximations. We illustrate the theory with some simulation results from Gaussian graphical models, and sparse principal component analysis.

arXiv comment53 pages, 3 figures

References in corpus (14)

• Simultaneous analysis of Lasso and Dantzig selector
• Spike and slab variable selection: Frequentist and Bayesian strategies
• An MCMC Approach to Classical Estimation
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Bayesian variable selection with shrinking and diffusing priors
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Frequentist Consistency of Variational Bayes
• Gibbs posterior for variable selection in high-dimensional classification and data mining
• Conditions for rapid mixing of parallel and simulated tempering on multimodal distributions
• Rapid Mixing of Hamiltonian Monte Carlo on Strongly Log-Concave Distributions
• Posterior convergence rates for estimating large precision matrices using graphical models
• Quasi-Bayesian analysis of nonparametric instrumental variables models
• Posterior consistency of nonparametric conditional moment restricted models
• Bayesian Estimation of Sparse Spiked Covariance Matrices in High Dimensions

Cited by in corpus (4)

• Accelerating proximal Markov chain Monte Carlo by using an explicit stabilised method
• Bayesian Regularization: From Tikhonov to Horseshoe
• A fast asynchronous MCMC sampler for sparse Bayesian inference
• Bayesian Analysis of High-dimensional Discrete Graphical Models