Model Selection for High-Dimensional Regression under the Generalized Irrepresentability Condition
arXiv:1305.0355
Abstract
In the high-dimensional regression model a response variable is linearly related to covariates, but the sample size is smaller than . We assume that only a small subset of covariates is `active' (i.e., the corresponding coefficients are non-zero), and consider the model-selection problem of identifying the active covariates. A popular approach is to estimate the regression coefficients through the Lasso (-regularized least squares). This is known to correctly identify the active set only if the irrelevant covariates are roughly orthogonal to the relevant ones, as quantified through the so called `irrepresentability' condition. In this paper we study the `Gauss-Lasso' selector, a simple two-stage method that first solves the Lasso, and then performs ordinary least squares restricted to the Lasso active set. We formulate `generalized irrepresentability condition' (GIC), an assumption that is substantially weaker than irrepresentability. We prove that, under GIC, the Gauss-Lasso correctly recovers the active set.
arXiv comment32 pages, 3 figures
References in corpus (15)
- Rejoinder to "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Simultaneous analysis of Lasso and Dantzig selector
- On the conditions used to prove oracle results for the Lasso
- Statistical significance in high-dimensional linear models
- Thresholded Lasso for high dimensional variable selection and statistical estimation
- Confidence Intervals for Low-Dimensional Parameters in High-Dimensional Linear Models
Cited by in corpus (4)
- Confidence intervals for high-dimensional inverse covariance estimation
- Swapping Variables for High-Dimensional Sparse Regression with Correlated Measurements
- Adaptive Feature Selection: Computationally Efficient Online Sparse Linear Regression under RIP
- Variable Selection via Adaptive False Negative Control in Linear Regression