Sparsity oracle inequalities for the Lasso

Abstract

This paper studies oracle properties of ${\ell }_1$-penalized least squares in nonparametric regression setting with random design. We show that the penalized least squares estimator satisfies sparsity oracle inequalities, i.e., bounds in terms of the number of non-zero components of the oracle vector. The results are valid even when the dimension of the model is (much) larger than the sample size and the regression matrix is not positive definite. They can be applied to high-dimensional linear regression, to nonparametric adaptive regression estimation and to the problem of aggregation of arbitrary estimators.

Published at http://dx.doi.org/10.1214/07-EJS008 in the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• High-dimensional generalized linear models and the lasso

Cited by in corpus (17)

• Nearly unbiased variable selection under minimax concave penalty
• Variable selection in nonparametric additive models
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Lasso-type recovery of sparse representations for high-dimensional data
• Performance guarantees for individualized treatment rules
• High-dimensional generalized linear models and the lasso
• Near-ideal model selection by ${\ell }_1$ minimization
• Least angle and ${\ell }_1$ penalized regression: A review
• The Dantzig selector and sparsity oracle inequalities
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Some sharp performance bounds for least squares regression with $L_1$ regularization
• Honest variable selection in linear and logistic regression models via ${\ell }_1$ and ${\ell }_1+{\ell }_2$ penalization
• Rejoinder: The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• Discussion: A tale of three cousins: Lasso, L2Boosting and Dantzig
• Pac-bayesian bounds for sparse regression estimation with exponential weights
• Sparse recovery in convex hulls via entropy penalization
• Structured, sparse regression with application to HIV drug resistance