High-dimensional generalized linear models and the lasso

Abstract

We consider high-dimensional generalized linear models with Lipschitz loss functions, and prove a nonasymptotic oracle inequality for the empirical risk minimizer with Lasso penalty. The penalty is based on the coefficients in the linear predictor, after normalization with the empirical norm. The examples include logistic regression, density estimation and classification with hinge loss. Least squares regression is also discussed.

Published in at http://dx.doi.org/10.1214/009053607000000929 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (4)

• High-dimensional graphs and variable selection with the Lasso
• Lasso-type recovery of sparse representations for high-dimensional data
• Sparsity oracle inequalities for the Lasso
• Best subset selection, persistence in high-dimensional statistical learning and optimization under $l_1$ constraint

Cited by in corpus (73)

• Nearly unbiased variable selection under minimax concave penalty
• Simultaneous analysis of Lasso and Dantzig selector
• On asymptotically optimal confidence regions and tests for high-dimensional models
• On the conditions used to prove oracle results for the Lasso
• Sure independence screening in generalized linear models with NP-dimensionality
• Least squares after model selection in high-dimensional sparse models
• Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming
• Variable selection in nonparametric additive models
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Lasso-type recovery of sparse representations for high-dimensional data
• High-dimensional additive modeling
• Performance guarantees for individualized treatment rules
• Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations
• Robust rank correlation based screening
• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
• Sparsity oracle inequalities for the Lasso
• Infinite Feature Selection: A Graph-based Feature Filtering Approach
• Sparse least trimmed squares regression for analyzing high-dimensional large data sets
• Statistical significance in high-dimensional linear models
• Least angle and ${\ell }_1$ penalized regression: A review
• L1-Penalization for Mixture Regression Models
• Sparse recovery under matrix uncertainty
• SCAD-penalized regression in high-dimensional partially linear models
• Least Ambiguous Set-Valued Classifiers with Bounded Error Levels
• Learning physical descriptors for materials science by compressed sensing
• Calibrating nonconvex penalized regression in ultra-high dimension
• Estimation for High-Dimensional Linear Mixed-Effects Models Using ${\ell }_1$-Penalization
• Lasso and probabilistic inequalities for multivariate point processes
• Pivotal estimation via square-root Lasso in nonparametric regression
• The Dantzig selector and sparsity oracle inequalities
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Variable selection in high-dimensional linear models: partially faithful distributions and the PC-simple algorithm
• Endogeneity in high dimensions
• Some sharp performance bounds for least squares regression with $L_1$ regularization
• The Lasso for High-Dimensional Regression with a Possible Change-Point
• Sparse Regression Learning by Aggregation and Langevin Monte-Carlo
• Oracle inequalities for the lasso in the Cox model
• A new perspective on least squares under convex constraint
• GLMMLasso: An Algorithm for High-Dimensional Generalized Linear Mixed Models Using L1-Penalization
• Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
• Honest variable selection in linear and logistic regression models via ${\ell }_1$ and ${\ell }_1+{\ell }_2$ penalization
• Consistent group selection in high-dimensional linear regression
• Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models
• Innovated interaction screening for high-dimensional nonlinear classification
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Factor models and variable selection in high-dimensional regression analysis
• Discussion: A tale of three cousins: Lasso, L2Boosting and Dantzig
• Oracle Estimation of a Change Point in High Dimensional Quantile Regression
• Mirror averaging with sparsity priors
• Asymptotic equivalence of regularization methods in thresholded parameter space
• SPADES and mixture models
• Sparse recovery in convex hulls via entropy penalization
• Covariate Selection in High-Dimensional Generalized Linear Models With Measurement Error
• Robust Parametric Classification and Variable Selection by a Minimum Distance Criterion
• Lasso type classifiers with a reject option
• On Clustering Categories of Categorical Predictors in Generalized Linear Models
• The log-linear group-lasso estimator and its asymptotic properties
• PAC-Bayesian Estimation and Prediction in Sparse Additive Models
• Quasi-Likelihood and/or Robust Estimation in High Dimensions
• Adaptive Bayesian density regression for high-dimensional data
• Estimating Treatment Effect under Additive Hazards Models with High-dimensional Covariates
• Smoothing ${\ell }_1$-penalized estimators for high-dimensional time-course data
• Structured Estimation in Nonparameteric Cox Model
• Greedy algorithms for prediction
• Fast and Robust Sparsity Learning over Networks: A Decentralized Surrogate Median Regression Approach
• General nonexact oracle inequalities for classes with a subexponential envelope
• DIF Analysis with Unknown Groups and Anchor Items
• Fairness-aware Model-agnostic Positive and Unlabeled Learning
• Parallel integrative learning for large-scale multi-response regression with incomplete outcomes
• Sample average approximation with heavier tails II: localization in stochastic convex optimization and persistence results for the Lasso
• On Hodges' Superefficiency and Merits of Oracle Property in Model Selection
• Ultra high dimensional generalized additive model: Unified Theory and Methods
• Selecting Penalty Parameters of High-Dimensional M-Estimators using Bootstrapping after Cross-Validation