Group Spike and Slab Variational Bayes

Abstract

We introduce Group Spike-and-slab Variational Bayes (GSVB), a scalable method for group sparse regression. A fast co-ordinate ascent variational inference (CAVI) algorithm is developed for several common model families including Gaussian, Binomial and Poisson. Theoretical guarantees for our proposed approach are provided by deriving contraction rates for the variational posterior in grouped linear regression. Through extensive numerical studies, we demonstrate that GSVB provides state-of-the-art performance, offering a computationally inexpensive substitute to MCMC, whilst performing comparably or better than existing MAP methods. Additionally, we analyze three real world datasets wherein we highlight the practical utility of our method, demonstrating that GSVB provides parsimonious models with excellent predictive performance, variable selection and uncertainty quantification.

References in corpus (11)

• Variational Inference: A Review for Statisticians
• Spike and slab variable selection: Frequentist and Bayesian strategies
• On asymptotically optimal confidence regions and tests for high-dimensional models
• Variable selection in nonparametric additive models
• A Selective Review of Group Selection in High-Dimensional Models
• Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors
• Bayesian linear regression with sparse priors
• Bayesian Variable Selection and Estimation for Group Lasso
• Variational Bayes for high-dimensional linear regression with sparse priors
• Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models
• Variational Bayes for high-dimensional proportional hazards models with applications within gene expression