Prediction error of cross-validated Lasso
arXiv:1502.06291
Abstract
In spite of the wealth of literature on the theoretical properties of the Lasso, there is very little known when the value of the tuning parameter is chosen using the data, even though this is what actually happens in practice. We give a general upper bound on the prediction error of Lasso when the tuning parameter is chosen using a variant of 2-fold cross-validation. No special assumption is made about the structure of the design matrix, and the tuning parameter is allowed to be optimized over an arbitrary data-dependent set of values. The proof is based on a general principle that may extend to other kinds of cross-validation as well as to other penalized regression methods. Based on this result, we propose a new estimate for error variance in high dimensional regression and prove that it has good properties under minimal assumptions.
arXiv comment18 pages. Minor revisions and additional references
References in corpus (18)
- Rejoinder to "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Simultaneous analysis of Lasso and Dantzig selector
- On the "degrees of freedom" of the lasso
- Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
- Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming
- A significance test for the lasso
- A new perspective on least squares under convex constraint
- Exponential Screening and optimal rates of sparse estimation
- Assumptionless consistency of the Lasso
Cited by in corpus (6)
- Estimating the error variance in a high-dimensional linear model
- Cross-Validation with Confidence
- A Survey of Tuning Parameter Selection for High-dimensional Regression
- High dimensional regression and matrix estimation without tuning parameters
- Using the LASSO for gene selection in bladder cancer data
- Automatic Double Machine Learning for Continuous Treatment Effects