On the Prediction Performance of the Lasso

Abstract

Although the Lasso has been extensively studied, the relationship between its prediction performance and the correlations of the covariates is not fully understood. In this paper, we give new insights into this relationship in the context of multiple linear regression. We show, in particular, that the incorporation of a simple correlation measure into the tuning parameter can lead to a nearly optimal prediction performance of the Lasso even for highly correlated covariates. However, we also reveal that for moderately correlated covariates, the prediction performance of the Lasso can be mediocre irrespective of the choice of the tuning parameter. We finally show that our results also lead to near-optimal rates for the least-squares estimator with total variation penalty.

References in corpus (9)

• Near-ideal model selection by ${\ell }_1$ minimization
• Sparsity oracle inequalities for the Lasso
• L1-Penalization for Mixture Regression Models
• Aggregation for Gaussian regression
• On risk bounds in isotonic and other shape restricted regression problems
• Consistencies and rates of convergence of jump-penalized least squares estimators
• Some sharp performance bounds for least squares regression with $L_1$ regularization
• A Practical Scheme and Fast Algorithm to Tune the Lasso With Optimality Guarantees
• Transductive versions of the LASSO and the Dantzig Selector

Cited by in corpus (36)

• Compressed Sensing with 1D Total Variation: Breaking Sample Complexity Barriers via Non-Uniform Recovery
• The DFS Fused Lasso: Linear-Time Denoising over General Graphs
• Vector-Valued Graph Trend Filtering with Non-Convex Penalties
• Estimating the error variance in a high-dimensional linear model
• Approximate Recovery in Changepoint Problems, from ${\ell }_2$ Estimation Error Rates
• Adaptive Risk Bounds in Univariate Total Variation Denoising and Trend Filtering
• Sharp oracle bounds for monotone and convex regression through aggregation
• Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation
• New Risk Bounds for 2D Total Variation Denoising
• Synthesis and analysis in total variation regularization
• Cross-Validation with Confidence
• On the total variation regularized estimator over a class of tree graphs
• Adaptive Estimation of Multivariate Piecewise Polynomials and Bounded Variation Functions by Optimal Decision Trees
• Tuning parameter calibration for prediction in personalized medicine
• Approximate $l_0$-penalized estimation of piecewise-constant signals on graphs
• Regression modeling on stratified data with the lasso
• Hierarchical Convex Optimization by the Hybrid Steepest Descent Method with Proximal Splitting Operators -- Enhancements of SVM and Lasso
• Joint estimation of $K$ related regression models with simple $L_1$-norm penalties
• Binarsity: a penalization for one-hot encoded features in linear supervised learning
• Optimistic lower bounds for convex regularized least-squares
• Estimating the Lasso's Effective Noise
• Regularization Effect of Fast Gradient Sign Method and its Generalization
• Element-wise estimation error of a total variation regularized estimator for change point detection
• Balancing Statistical and Computational Precision: A General Theory and Applications to Sparse Regression
• Sparse Partially Linear Additive Models
• On a phase transition in general order spline regression
• Iterative Alpha Expansion for estimating gradient-sparse signals from linear measurements
• Frame-constrained Total Variation Regularization for White Noise Regression
• Oracle inequalities for square root analysis estimators with application to total variation penalties
• Aggregation of supports along the Lasso path
• Maximum Regularized Likelihood Estimators: A General Prediction Theory and Applications
• Optimal prediction for sparse linear models? Lower bounds for coordinate-separable M-estimators
• Debiased and threshold ridge regression for linear model with heteroskedastic and dependent error
• Estimating Piecewise Monotone Signals
• Sharp Oracle Inequalities for Low-complexity Priors
• Tuning parameter calibration for ${\ell }_1$-regularized logistic regression