Simultaneous Variable and Covariance Selection with the Multivariate Spike-and-Slab Lasso

Abstract

We propose a Bayesian procedure for simultaneous variable and covariance selection using continuous spike-and-slab priors in multivariate linear regression models where q possibly correlated responses are regressed onto p predictors. Rather than relying on a stochastic search through the high-dimensional model space, we develop an ECM algorithm similar to the EMVS procedure of Rockova & George (2014) targeting modal estimates of the matrix of regression coefficients and residual precision matrix. Varying the scale of the continuous spike densities facilitates dynamic posterior exploration and allows us to filter out negligible regression coefficients and partial covariances gradually. Our method is seen to substantially outperform regularization competitors on simulated data. We demonstrate our method with a re-examination of data from a recent observational study of the effect of playing high school football on several later-life cognition, psychological, and socio-economic outcomes.

References in corpus (6)

• Nearly unbiased variable selection under minimax concave penalty
• Pathwise coordinate optimization
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• A sparse conditional Gaussian graphical model for analysis of genetical genomics data
• Scaling It Up: Stochastic Search Structure Learning in Graphical Models
• An Expectation Conditional Maximization approach for Gaussian graphical models

Cited by in corpus (15)

• Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models
• Spike-and-Slab Meets LASSO: A Review of the Spike-and-Slab LASSO
• BayesSUR: An R package for high-dimensional multivariate Bayesian variable and covariance selection in linear regression
• The Graphical Horseshoe Estimator for Inverse Covariance Matrices
• Scalable high-dimensional Bayesian varying coefficient models with unknown within-subject covariance
• Bayesian Bootstrap Spike-and-Slab LASSO
• Joint Mean-Covariance Estimation via the Horseshoe with an Application in Genomic Data Analysis
• Bayesian Joint Spike-and-Slab Graphical Lasso
• Bayesian high-dimensional covariate selection in non-linear mixed-effects models using the SAEM algorithm
• Horseshoe Regularization for Machine Learning in Complex and Deep Models
• The EAS approach to variable selection for multivariate response data in high-dimensional settings
• Covariate-Assisted Bayesian Graph Learning for Heterogeneous Data
• Ultra High-dimensional Multivariate Posterior Contraction Rate Under Shrinkage Priors
• For high-dimensional hierarchical models, consider exchangeability of effects across covariates instead of across datasets
• Bayesian Regularization for Graphical Models with Unequal Shrinkage