On the "degrees of freedom" of the lasso

Abstract

We study the effective degrees of freedom of the lasso in the framework of Stein's unbiased risk estimation (SURE). We show that the number of nonzero coefficients is an unbiased estimate for the degrees of freedom of the lasso--a conclusion that requires no special assumption on the predictors. In addition, the unbiased estimator is shown to be asymptotically consistent. With these results on hand, various model selection criteria--$C_p$, AIC and BIC--are available, which, along with the LARS algorithm, provide a principled and efficient approach to obtaining the optimal lasso fit with the computational effort of a single ordinary least-squares fit.

Published in at http://dx.doi.org/10.1214/009053607000000127 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery

Cited by in corpus (63)

• On the adaptive elastic-net with a diverging number of parameters
• The composite absolute penalties family for grouped and hierarchical variable selection
• A proximal iteration for deconvolving Poisson noisy images using sparse representations
• Least angle and ${\ell }_1$ penalized regression: A review
• Estimation for High-Dimensional Linear Mixed-Effects Models Using ${\ell }_1$-Penalization
• Sparse logistic principal components analysis for binary data
• Penalized model-based clustering with cluster-specific diagonal covariance matrices and grouped variables
• Statistical Significance of the Netflix Challenge
• Penalized variable selection procedure for Cox models with semiparametric relative risk
• The Lasso with general Gaussian designs with applications to hypothesis testing
• Regularized 3D functional regression for brain image data via Haar wavelets
• Comment: Boosting Algorithms: Regularization, Prediction and Model Fitting
• Smoothed Bernstein Online Aggregation for Day-Ahead Electricity Demand Forecasting
• On Improved Loss Estimation for Shrinkage Estimators
• Regularization with the Smooth-Lasso procedure
• A neural network model for solvency calculations in life insurance
• Learning networks from high dimensional binary data: An application to genomic instability data
• Variable selection and estimation for semi-parametric multiple-index models
• Algorithms for Fitting the Constrained Lasso
• Parameter Choices for Sparse Regularization with the ${\ell }_1$ Norm
• Sparse estimation via nonconcave penalized likelihood in a factor analysis model
• A Century of Economic Policy Uncertainty Through the French-Canadian Lens
• Inferring Signaling Pathways with Probabilistic Programming
• Prediction Errors for Penalized Regressions based on Generalized Approximate Message Passing
• Real Interference Alignment
• Regression shrinkage and grouping of highly correlated predictors with HORSES
• Penalized Likelihood Regression in Reproducing Kernel Hilbert Spaces with Randomized Covariate Data
• Investigating Competition in Financial Markets: A Sparse Autologistic Model for Dynamic Network Data
• Multiple-Instance Logistic Regression with LASSO Penalty
• Mixture of von Mises-Fisher distribution with sparse prototypes
• Mixture Components Inference for Sparse Regression: Introduction and Application for Estimation of Neuronal Signal from fMRI BOLD
• Adding smoothing splines to the SAM model improves stock assessment
• Estimating the Rate Constant from Biosensor Data via an Adaptive Variational Bayesian Approach
• Variance Estimation Using Refitted Cross-validation in Ultrahigh Dimensional Regression
• Blockwise and coordinatewise thresholding to combine tests of different natures in modern ANOVA
• The Degrees of Freedom of the Group Lasso
• Credit scoring using neural networks and SURE posterior probability calibration
• Automatic Hyperparameter Tuning in Sparse Matrix Factorization
• Estimation of oblique structure via penalized likelihood factor analysis
• Graph estimation with joint additive models
• Excess Optimism: How Biased is the Apparent Error of an Estimator Tuned by SURE?
• Asymmetric Degrees of Freedom of the Full-Duplex MIMO 3-Way Channel with Unicast and Broadcast Messages
• Sparse Correspondence Analysis for Contingency Tables
• Semi-supervised linear regression: enhancing efficiency and robustness in high dimensions
• Partial Correlation Estimation by Joint Sparse Regression Models
• Revisiting minimum description length complexity in overparameterized models
• Weak Signal Identification and Inference in Penalized Model Selection
• Capacity of a Class of Symmetric SIMO Gaussian Interference Channels within O(1)
• Spatial point processes intensity estimation with a diverging number of covariates
• Spatially Clustered Regression
• Generating Probabilities From Numerical Weather Forecasts by Logistic Regression
• Prediction Weighted Maximum Frequency Selection
• Estimating the Penalty Level of ${\ell }_1$-minimization via Two Gaussian Approximation Methods
• Computing the degrees of freedom of rank-regularized estimators and cousins
• yaglm: a Python package for fitting and tuning generalized linear models that supports structured, adaptive and non-convex penalties
• Recovering lost and absent information in temporal networks
• Regularization for the Approximation of 2D Set of Points via the Length of the Curve
• Assessment of Case Influence in the Lasso with a Case-weight Adjusted Solution Path
• gLOP: the global and Local Penalty for Capturing Predictive Heterogeneity
• Model-based clustering of multivariate binary data with dimension reduction
• Weak convergence of the regularization path in penalized M-estimation
• A non-convex regularization approach for stable estimation of loss development factors
• Estimation and Selection Properties of the LAD Fused Lasso Signal Approximator