Nearly optimal Bayesian Shrinkage for High Dimensional Regression

Abstract

During the past decade, shrinkage priors have received much attention in Bayesian analysis of high-dimensional data. This paper establishes the posterior consistency for high-dimensional linear regression with a class of shrinkage priors, which has a heavy and flat tail and allocates a sufficiently large probability mass in a very small neighborhood of zero. While enjoying its efficiency in posterior simulations, the shrinkage prior can lead to a nearly optimal posterior contraction rate and variable selection consistency as the spike-and-slab prior. Our numerical results show that under the posterior consistency, Bayesian methods can yield much better results in variable selection than the regularization methods such as Lasso and SCAD. This paper also establishes a Bernstein von-Mises type result, which leads to a convenient way of uncertainty quantification for regression coefficient estimates.

References in corpus (5)

• Nearly unbiased variable selection under minimax concave penalty
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Sure Screening for Gaussian Graphical Models
• Bayesian Sparse Global-Local Shrinkage Regression for Selection of Grouped Variables

Cited by in corpus (18)

• On the Beta Prime Prior for Scale Parameters in High-Dimensional Bayesian Regression Models
• Variational Inference in high-dimensional linear regression
• Neuronized Priors for Bayesian Sparse Linear Regression
• Consistent Sparse Deep Learning: Theory and Computation
• Nearly Optimal Variational Inference for High Dimensional Regression with Shrinkage Priors
• Bayesian Linear Regression for Multivariate Responses Under Group Sparsity
• Optimal False Discovery Control of Minimax Estimator
• Sketching in Bayesian High Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors
• Bayesian high-dimensional linear regression with generic spike-and-slab priors
• Graph signal denoising using $t$-shrinkage priors
• Adaptive Variational Bayesian Inference for Sparse Deep Neural Network
• Strategic Bayesian Asset Allocation
• Time Fused Coefficient SIR Model with Application to COVID-19 Epidemic in the United States
• Bayesian High-dimensional Semi-parametric Inference beyond sub-Gaussian Errors
• Bayesian Factor-adjusted Sparse Regression
• Bayesian Group Selection in Logistic Regression with Application to MRI Data Analysis
• Bayesian $L_{\frac{1}{2}}$ regression
• Posterior Inference for Quantile Regression: Adaptation to Sparsity