An elementary analysis of ridge regression with random design

Abstract

In this note, we provide an elementary analysis of the prediction error of ridge regression with random design. The proof is short and self-contained. In particular, it bypasses the use of Rudelson's deviation inequality for covariance matrices, through a combination of exchangeability arguments, matrix perturbation and operator convexity.

fixes a typo, small changes; 9 pages

References in corpus (5)

• Benign Overfitting in Linear Regression
• Parallelizing Stochastic Gradient Descent for Least Squares Regression: mini-batching, averaging, and model misspecification
• Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices
• Performance of empirical risk minimization in linear aggregation
• Learning with incremental iterative regularization