Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models

Abstract

We introduce the spike-and-slab group lasso (SSGL) for Bayesian estimation and variable selection in linear regression with grouped variables. We further extend the SSGL to sparse generalized additive models (GAMs), thereby introducing the first nonparametric variant of the spike-and-slab lasso methodology. Our model simultaneously performs group selection and estimation, while our fully Bayes treatment of the mixture proportion allows for model complexity control and automatic self-adaptivity to different levels of sparsity. We develop theory to uniquely characterize the global posterior mode under the SSGL and introduce a highly efficient block coordinate ascent algorithm for maximum a posteriori (MAP) estimation. We further employ de-biasing methods to provide uncertainty quantification of our estimates. Thus, implementation of our model avoids the computational intensiveness of Markov chain Monte Carlo (MCMC) in high dimensions. We derive posterior concentration rates for both grouped linear regression and sparse GAMs when the number of covariates grows at nearly exponential rate with sample size. Finally, we illustrate our methodology through extensive simulations and data analysis.

67 pages. 9 figures, 5 tables. Additional details for the data analysis were added to Section 8 and Appendix C

References in corpus (8)

• High-dimensional graphs and variable selection with the Lasso
• On asymptotically optimal confidence regions and tests for high-dimensional models
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• A Selective Review of Group Selection in High-Dimensional Models
• Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors
• Bayesian Variable Selection and Estimation for Group Lasso
• Simultaneous Variable and Covariance Selection with the Multivariate Spike-and-Slab Lasso
• Variance prior forms for high-dimensional Bayesian variable selection

Cited by in corpus (4)

• Multivariate Bayesian structured variable selection for pharmacogenomic studies
• Spike-and-Slab LASSO Generalized Additive Models and Scalable Algorithms for High-Dimensional Data Analysis
• The EAS approach to variable selection for multivariate response data in high-dimensional settings
• Group Spike and Slab Variational Bayes