A Practical Scheme and Fast Algorithm to Tune the Lasso With Optimality Guarantees

Abstract

We introduce a novel scheme for choosing the regularization parameter in high-dimensional linear regression with Lasso. This scheme, inspired by Lepski's method for bandwidth selection in non-parametric regression, is equipped with both optimal finite-sample guarantees and a fast algorithm. In particular, for any design matrix such that the Lasso has low sup-norm error under an "oracle choice" of the regularization parameter, we show that our method matches the oracle performance up to a small constant factor, and show that it can be implemented by performing simple tests along a single Lasso path. By applying the Lasso to simulated and real data, we find that our novel scheme can be faster and more accurate than standard schemes such as Cross-Validation.

References in corpus (2)

• Nearly unbiased variable selection under minimax concave penalty
• Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator

Cited by in corpus (18)

• On the Prediction Performance of the Lasso
• Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularization
• Rank-one Convexification for Sparse Regression
• Robust Information Criterion for Model Selection in Sparse High-Dimensional Linear Regression Models
• Tuning parameter calibration for prediction in personalized medicine
• Inference for high-dimensional instrumental variables regression
• A Survey of Tuning Parameter Selection for High-dimensional Regression
• Estimating the Lasso's Effective Noise
• High dimensional regression and matrix estimation without tuning parameters
• Balancing Statistical and Computational Precision: A General Theory and Applications to Sparse Regression
• False Discovery Rates in Biological Networks
• Tuning-free ridge estimators for high-dimensional generalized linear models
• The False Positive Control Lasso
• OLR-WA: Online Weighted Average Linear Regression in Multivariate Data Streams
• Provable Training Set Debugging for Linear Regression
• On the use of cross-validation for the calibration of the adaptive lasso
• Thresholding Greedy Pursuit for Sparse Recovery Problems
• Tuning parameter calibration for ${\ell }_1$-regularized logistic regression