Spike-and-Slab Meets LASSO: A Review of the Spike-and-Slab LASSO

Abstract

High-dimensional data sets have become ubiquitous in the past few decades, often with many more covariates than observations. In the frequentist setting, penalized likelihood methods are the most popular approach for variable selection and estimation in high-dimensional data. In the Bayesian framework, spike-and-slab methods are commonly used as probabilistic constructs for high-dimensional modeling. Within the context of linear regression, Rockova and George (2018) introduced the spike-and-slab LASSO (SSL), an approach based on a prior which provides a continuum between the penalized likelihood LASSO and the Bayesian point-mass spike-and-slab formulations. Since its inception, the spike-and-slab LASSO has been extended to a variety of contexts, including generalized linear models, factor analysis, graphical models, and nonparametric regression. The goal of this paper is to survey the landscape surrounding spike-and-slab LASSO methodology. First we elucidate the attractive properties and the computational tractability of SSL priors in high dimensions. We then review methodological developments of the SSL and outline several theoretical developments. We illustrate the methodology on both simulated and real datasets.

34 pages, 2 tables, 3 figures. Section 3.3 was added to illustrate the method

References in corpus (5)

• Nearly unbiased variable selection under minimax concave penalty
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Bayesian variable selection with shrinking and diffusing priors
• Bayesian Bootstrap Spike-and-Slab LASSO
• Adaptive Bayesian SLOPE -- High-dimensional Model Selection with Missing Values

Cited by in corpus (6)

• The Reciprocal Bayesian LASSO
• Sparse online variational Bayesian regression
• Spike-and-Slab LASSO Generalized Additive Models and Scalable Algorithms for High-Dimensional Data Analysis
• Bayesian high-dimensional covariate selection in non-linear mixed-effects models using the SAEM algorithm
• Information Directed Sampling for Sparse Linear Bandits
• Screening the Discrepancy Function of a Computer Model