Bayesian iterative screening in ultra-high dimensional linear regressions

Abstract

Variable selection in ultra-high dimensional linear regression is often preceded by a screening step to significantly reduce the dimension. Here we develop a Bayesian variable screening method (BITS) guided by the posterior model probabilities. BITS can successfully integrate prior knowledge, if any, on effect sizes, and the number of true variables. BITS iteratively includes potential variables with the highest posterior probability accounting for the already selected variables. It is implemented by a fast Cholesky update algorithm and is shown to have the screening consistency property. BITS is built based on a model with Gaussian errors, yet, the screening consistency is proved to hold under more general tail conditions. The notion of posterior screening consistency allows the resulting model to provide a good starting point for further Bayesian variable selection methods. A new screening consistent stopping rule based on posterior probability is developed. Simulation studies and real data examples are used to demonstrate scalability and fine screening performance.

References in corpus (7)

• Spike and slab variable selection: Frequentist and Bayesian strategies
• Sure independence screening in generalized linear models with NP-dimensionality
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data
• Bayesian variable selection with shrinking and diffusing priors
• High-dimensional variable selection via tilting
• Determinantal Point Process Priors for Bayesian Variable Selection in Linear Regression