High-dimensional variable selection

Abstract

This paper explores the following question: what kind of statistical guarantees can be given when doing variable selection in high-dimensional models? In particular, we look at the error rates and power of some multi-stage regression methods. In the first stage we fit a set of candidate models. In the second stage we select one model by cross-validation. In the third stage we use hypothesis testing to eliminate some variables. We refer to the first two stages as "screening" and the last stage as "cleaning." We consider three screening methods: the lasso, marginal regression, and forward stepwise regression. Our method gives consistent variable selection under certain conditions.

Published in at http://dx.doi.org/10.1214/08-AOS646 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (16)

• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• High-dimensional graphs and variable selection with the Lasso
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Lasso-type recovery of sparse representations for high-dimensional data
• Approximation and learning by greedy algorithms
• Boosting for high-dimensional linear models
• Sharp thresholds for high-dimensional and noisy recovery of sparsity

Cited by in corpus (48)

• On asymptotically optimal confidence regions and tests for high-dimensional models
• A significance test for the lasso
• SLOPE - Adaptive variable selection via convex optimization
• Statistical significance in high-dimensional linear models
• High-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdi
• Estimation for High-Dimensional Linear Mixed-Effects Models Using ${\ell }_1$-Penalization
• Confidence intervals for high-dimensional inverse covariance estimation
• Variable selection in high-dimensional linear models: partially faithful distributions and the PC-simple algorithm
• Endogeneity in high dimensions
• Honest variable selection in linear and logistic regression models via ${\ell }_1$ and ${\ell }_1+{\ell }_2$ penalization
• High-dimensional inference in misspecified linear models
• Goodness of fit tests for high-dimensional linear models
• UPS delivers optimal phase diagram in high-dimensional variable selection
• Covariate assisted screening and estimation
• Tight conditions for consistency of variable selection in the context of high dimensionality
• Significance tests of feature relevance for a black-box learner
• Convex and non-convex regularization methods for spatial point processes intensity estimation
• Genomics models in radiotherapy: from mechanistic to machine learning
• Hierarchical inference for genome-wide association studies: a view on methodology with software
• A Two-Sample Conditional Distribution Test Using Conformal Prediction and Weighted Rank Sum
• Discussion: "A significance test for the lasso"
• Sparse Representation Classification Beyond L1 Minimization and the Subspace Assumption
• Controlling the False Discovery Rate in Transformational Sparsity: Split Knockoffs
• A Hierarchical Integrative Group LASSO (HiGLASSO) Framework for Analyzing Environmental Mixtures
• Discussion: "A significance test for the lasso"
• Selective conformal inference with false coverage-statement rate control
• The geometry of least squares in the 21st century
• Solving FDR-Controlled Sparse Regression Problems with Five Million Variables on a Laptop
• Combining exchangeable p-values
• On Selecting and Conditioning in Multiple Testing and Selective Inference
• Predictive Quantile Regression with Mixed Roots and Increasing Dimensions: The ALQR Approach
• Feature-specific inference for penalized regression using local false discovery rates
• Effect modification in anchored indirect treatment comparisons: Comments on "Matching-adjusted indirect comparisons: Application to time-to-event data"
• Two-Stage Robust and Sparse Distributed Statistical Inference for Large-Scale Data
• Another Look at Bandwidth-free Inference: a Sample Splitting Approach
• ${\ell }_1$-penalized Multinomial Regression: Estimation, inference, and prediction, with an application to risk factor identification for different dementia subtypes
• Analyzing Risk Factors for Post-Acute Recovery in Older Adults with Alzheimer's Disease and Related Dementia: A New Semi-Parametric Model for Large-Scale Medicare Claims
• High-Dimensional Screening Using Multiple Grouping of Variables
• Estimator Augmentation with Applications in High-Dimensional Group Inference
• A Metropolized adaptive subspace algorithm for high-dimensional Bayesian variable selection
• Conformal inference for random objects
• Marginal false discovery rates for penalized regression models
• Discussion: "A significance test for the lasso"
• Exact model comparisons in the plausibility framework
• Neural interval-censored survival regression with feature selection
• Post-selection Inference in Regression Models for Group Testing Data
• Variable Selection in Functional Linear Cox Model
• KL-BSS: Rethinking optimality for neighbourhood selection in structural equation models