Bayesian variable selection with shrinking and diffusing priors

Abstract

We consider a Bayesian approach to variable selection in the presence of high dimensional covariates based on a hierarchical model that places prior distributions on the regression coefficients as well as on the model space. We adopt the well-known spike and slab Gaussian priors with a distinct feature, that is, the prior variances depend on the sample size through which appropriate shrinkage can be achieved. We show the strong selection consistency of the proposed method in the sense that the posterior probability of the true model converges to one even when the number of covariates grows nearly exponentially with the sample size. This is arguably the strongest selection consistency result that has been available in the Bayesian variable selection literature; yet the proposed method can be carried out through posterior sampling with a simple Gibbs sampler. Furthermore, we argue that the proposed method is asymptotically similar to model selection with the $L_0$ penalty. We also demonstrate through empirical work the fine performance of the proposed approach relative to some state of the art alternatives.

Published in at http://dx.doi.org/10.1214/14-AOS1207 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (4)

• Bayesian empirical likelihood for quantile regression
• Consistency of objective Bayes factors as the model dimension grows
• Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
• Asymptotic oracle properties of SCAD-penalized least squares estimators

Cited by in corpus (2)

• Risk and Regret of Hierarchical Bayesian Learners
• On Joint Estimation of Gaussian Graphical Models for Spatial and Temporal Data