Spike and slab variational Bayes for high dimensional logistic regression
arXiv:2010.11665
Abstract
Variational Bayes (VB) is a popular scalable alternative to Markov chain Monte Carlo for Bayesian inference. We study a mean-field spike and slab VB approximation of widely used Bayesian model selection priors in sparse high-dimensional logistic regression. We provide non-asymptotic theoretical guarantees for the VB posterior in both and prediction loss for a sparse truth, giving optimal (minimax) convergence rates. Since the VB algorithm does not depend on the unknown truth to achieve optimality, our results shed light on effective prior choices. We confirm the improved performance of our VB algorithm over common sparse VB approaches in a numerical study.
arXiv commentNeurIPS 2020. Several typos corrected
References in corpus (9)
- Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
- Frequentist Consistency of Variational Bayes
- Variational Bayesian Inference with Stochastic Search
- Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
- Adaptive Bernstein-von Mises theorems in Gaussian white noise
- Posterior Concentration for Sparse Deep Learning
- A Variational Algorithm for Bayesian Variable Selection
- Bayesian Joint Spike-and-Slab Graphical Lasso
- Methods for Bayesian Variable Selection with Binary Response Data using the EM Algorithm