Model Selection for High-Dimensional Regression under the Generalized Irrepresentability Condition

Abstract

In the high-dimensional regression model a response variable is linearly related to $p$ covariates, but the sample size $n$ is smaller than $p$. We assume that only a small subset of covariates is `active' (i.e., the corresponding coefficients are non-zero), and consider the model-selection problem of identifying the active covariates. A popular approach is to estimate the regression coefficients through the Lasso (${\ell }_1$-regularized least squares). This is known to correctly identify the active set only if the irrelevant covariates are roughly orthogonal to the relevant ones, as quantified through the so called `irrepresentability' condition. In this paper we study the `Gauss-Lasso' selector, a simple two-stage method that first solves the Lasso, and then performs ordinary least squares restricted to the Lasso active set. We formulate `generalized irrepresentability condition' (GIC), an assumption that is substantially weaker than irrepresentability. We prove that, under GIC, the Gauss-Lasso correctly recovers the active set.

32 pages, 3 figures

References in corpus (3)

• Statistical significance in high-dimensional linear models
• Thresholded Lasso for high dimensional variable selection and statistical estimation
• Confidence Intervals for Low-Dimensional Parameters in High-Dimensional Linear Models

Cited by in corpus (4)

• Confidence intervals for high-dimensional inverse covariance estimation
• Swapping Variables for High-Dimensional Sparse Regression with Correlated Measurements
• Adaptive Feature Selection: Computationally Efficient Online Sparse Linear Regression under RIP
• Variable Selection via Adaptive False Negative Control in Linear Regression