Empirical Bayes posterior concentration in sparse high-dimensional linear models

Abstract

We propose a new empirical Bayes approach for inference in the $p\gg n$ normal linear model. The novelty is the use of data in the prior in two ways, for centering and regularization. Under suitable sparsity assumptions, we establish a variety of concentration rate results for the empirical Bayes posterior distribution, relevant for both estimation and model selection. Computation is straightforward and fast, and simulation results demonstrate the strong finite-sample performance of the empirical Bayes model selection procedure.

24 pages, 3 tables, and 3 extra pages to correct a couple minor mistakes in the published version

References in corpus (8)

• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Bayesian linear regression with sparse priors
• Bayesian variable selection with shrinking and diffusing priors
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Gibbs posterior for variable selection in high-dimensional classification and data mining
• Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
• On rates of convergence for posterior distributions in infinite-dimensional models
• A General Framework for Bayes Structured Linear Models

Cited by in corpus (12)

• Variational Bayes for high-dimensional linear regression with sparse priors
• A comparison of learning rate selection methods in generalized Bayesian inference
• Gibbs posterior concentration rates under sub-exponential type losses
• Generalized Geographically Weighted Regression Model within a Modularized Bayesian Framework
• Data-driven priors and their posterior concentration rates
• Bayesian Sparse Linear Regression with Unknown Symmetric Error
• Empirical priors and coverage of posterior credible sets in a sparse normal mean model
• Concentration of posterior probabilities and normalized L0 criteria
• Empirical priors and posterior concentration rates for a monotone density
• Empirical Bayes inference in sparse high-dimensional generalized linear models
• Uncertainty Quantification for High Dimensional Sparse Nonparametric Additive Models
• Generalized Bayes Approach to Inverse Problems with Model Misspecification