High-dimensional generalized linear models and the lasso

Abstract

We consider high-dimensional generalized linear models with Lipschitz loss functions, and prove a nonasymptotic oracle inequality for the empirical risk minimizer with Lasso penalty. The penalty is based on the coefficients in the linear predictor, after normalization with the empirical norm. The examples include logistic regression, density estimation and classification with hinge loss. Least squares regression is also discussed.

Published in at http://dx.doi.org/10.1214/009053607000000929 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (3)

• Lasso-type recovery of sparse representations for high-dimensional data
• Sparsity oracle inequalities for the Lasso
• Best subset selection, persistence in high-dimensional statistical learning and optimization under $l_1$ constraint

Cited by in corpus (14)

• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Lasso-type recovery of sparse representations for high-dimensional data
• Sparsity oracle inequalities for the Lasso
• Least angle and ${\ell }_1$ penalized regression: A review
• SCAD-penalized regression in high-dimensional partially linear models
• The Dantzig selector and sparsity oracle inequalities
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Some sharp performance bounds for least squares regression with $L_1$ regularization
• Honest variable selection in linear and logistic regression models via ${\ell }_1$ and ${\ell }_1+{\ell }_2$ penalization
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Discussion: A tale of three cousins: Lasso, L2Boosting and Dantzig
• Sparse recovery in convex hulls via entropy penalization
• Lasso type classifiers with a reject option
• Smoothing ${\ell }_1$-penalized estimators for high-dimensional time-course data