Empirical priors for prediction in sparse high-dimensional linear regression

Abstract

In this paper we adopt the familiar sparse, high-dimensional linear regression model and focus on the important but often overlooked task of prediction. In particular, we consider a new empirical Bayes framework that incorporates data in the prior in two ways: one is to center the prior for the non-zero regression coefficients and the other is to provide some additional regularization. We show that, in certain settings, the asymptotic concentration of the proposed empirical Bayes posterior predictive distribution is very fast, and we establish a Bernstein--von Mises theorem which ensures that the derived empirical Bayes prediction intervals achieve the targeted frequentist coverage probability. The empirical prior has a convenient conjugate form, so posterior computations are relatively simple and fast. Finally, our numerical results demonstrate the proposed method's strong finite-sample performance in terms of prediction accuracy, uncertainty quantification, and computation time compared to existing Bayesian methods.

arXiv comment29 pages, 1 figure, 7 tables. Comments welcome at https://www.researchers.one/article/2019-03-1

References in corpus (8)

• Calibrating general posterior credible regions
• Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
• Conditional predictive inference post model selection
• Exact minimax estimation of the predictive density in sparse Gaussian models
• On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding
• An empirical $G$-Wishart prior for sparse high-dimensional Gaussian graphical models
• Bayesian inference in high-dimensional linear models using an empirical correlation-adaptive prior
• Posterior Concentration Properties of a General Class of Shrinkage Priors around Nearly Black Vectors

Cited by in corpus (5)

• Variational Bayes for high-dimensional linear regression with sparse priors
• A comparison of learning rate selection methods in generalized Bayesian inference
• Data-driven priors and their posterior concentration rates
• An empirical $G$-Wishart prior for sparse high-dimensional Gaussian graphical models
• Empirical Bayes inference in sparse high-dimensional generalized linear models