Bayesian variable selection with shrinking and diffusing priors

Abstract

We consider a Bayesian approach to variable selection in the presence of high dimensional covariates based on a hierarchical model that places prior distributions on the regression coefficients as well as on the model space. We adopt the well-known spike and slab Gaussian priors with a distinct feature, that is, the prior variances depend on the sample size through which appropriate shrinkage can be achieved. We show the strong selection consistency of the proposed method in the sense that the posterior probability of the true model converges to one even when the number of covariates grows nearly exponentially with the sample size. This is arguably the strongest selection consistency result that has been available in the Bayesian variable selection literature; yet the proposed method can be carried out through posterior sampling with a simple Gibbs sampler. Furthermore, we argue that the proposed method is asymptotically similar to model selection with the $L_0$ penalty. We also demonstrate through empirical work the fine performance of the proposed approach relative to some state of the art alternatives.

Published in at http://dx.doi.org/10.1214/14-AOS1207 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (4)

• Bayesian empirical likelihood for quantile regression
• Consistency of objective Bayes factors as the model dimension grows
• Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
• Asymptotic oracle properties of SCAD-penalized least squares estimators

Cited by in corpus (19)

• Efficient estimation and correction of selection-induced bias with order statistics
• Covariate-Assisted Bayesian Graph Learning for Heterogeneous Data
• A Bayesian Approach to Joint Estimation of Multiple Graphical Models
• Bayesian high-dimensional linear regression with generic spike-and-slab priors
• Risk and Regret of Hierarchical Bayesian Learners
• Scalable Bayesian change point detection with spike and slab priors
• Bayesian equation selection on sparse data for discovery of stochastic dynamical systems
• A robust Bayesian analysis of variable selection under prior ignorance
• Consistent Bayesian Sparsity Selection for High-dimensional Gaussian DAG Models with Multiplicative and Beta-mixture Priors
• Posterior consistency in multi-response regression models with non-informative priors for the error covariance matrix in growing dimensions
• On Posterior consistency of Bayesian Changepoint models
• Bayesian Edge Regression in Undirected Graphical Models to Characterize Interpatient Heterogeneity in Cancer
• Bayesian High-dimensional Semi-parametric Inference beyond sub-Gaussian Errors
• Bayesian Group Selection in Logistic Regression with Application to MRI Data Analysis
• Bayesian Factor-adjusted Sparse Regression
• Covariate-Elaborated Robust Partial Information Transfer with Conditional Spike-and-Slab Prior
• Revisiting High Dimensional Bayesian Model Selection for Gaussian Regression
• On Joint Estimation of Gaussian Graphical Models for Spatial and Temporal Data
• Adaptive posterior convergence in sparse high dimensional clipped generalized linear models