Bayesian linear regression with sparse priors

Abstract

We study full Bayesian procedures for high-dimensional linear regression under sparsity constraints. The prior is a mixture of point masses at zero and continuous distributions. Under compatibility conditions on the design matrix, the posterior distribution is shown to contract at the optimal rate for recovery of the unknown sparse vector, and to give optimal prediction of the response vector. It is also shown to select the correct sparse model, or at least the coefficients that are significantly different from zero. The asymptotic shape of the posterior distribution is characterized and employed to the construction and study of credible sets for uncertainty quantification.

Published at http://dx.doi.org/10.1214/15-AOS1334 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (7)

• Nearly unbiased variable selection under minimax concave penalty
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Bayesian linear regression with sparse priors
• Sparsity oracle inequalities for the Lasso
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators

Cited by in corpus (28)

• Bayesian linear regression with sparse priors
• Contributed Discussion to Uncertainty Quantification for the Horseshoe by Stéphanie van der Pas, Botond Szabó and Aad van der Vaart
• Empirical Bayes posterior concentration in sparse high-dimensional linear models
• Variational Bayes for high-dimensional linear regression with sparse priors
• Adaptive Bernstein-von Mises theorems in Gaussian white noise
• High-Dimensional Multivariate Posterior Consistency Under Global-Local Shrinkage Priors
• Conditions for Posterior Contraction in the Sparse Normal Means Problem
• On the Beta Prime Prior for Scale Parameters in High-Dimensional Bayesian Regression Models
• Nonparametric Statistical Inference via Metric Distribution Function in Metric Spaces
• Bayesian Inverse Uncertainty Quantification of the Physical Model Parameters for the Spallation Neutron Source First Target Station
• Bayesian Sparse Linear Regression with Unknown Symmetric Error
• Bayesian Effect Selection in Structured Additive Distributional Regression Models
• Group Inverse-Gamma Gamma Shrinkage for Sparse Regression with Block-Correlated Predictors
• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• Random weighting in LASSO regression
• Concentration of posterior probabilities and normalized L0 criteria
• The matryoshka doll prior: principled multiplicity correction in Bayesian model comparison
• Variable Selection with ABC Bayesian Forests
• Adaptive variational Bayes: Optimality, computation and applications
• Sparse online variational Bayesian regression
• Adaptive group-regularized logistic elastic net regression
• Empirical Bayes inference in sparse high-dimensional generalized linear models
• Misclassification bounds for PAC-Bayesian sparse deep learning
• On frequentist coverage errors of Bayesian credible sets in moderately high dimensions
• Approximate Laplace approximations for scalable model selection
• Posterior consistency in multi-response regression models with non-informative priors for the error covariance matrix in growing dimensions
• Group Spike and Slab Variational Bayes
• Covariate-Elaborated Robust Partial Information Transfer with Conditional Spike-and-Slab Prior