Risk Comparisons in Linear Regression: Implicit Regularization Dominates Explicit Regularization

Jingfeng Wu, Peter L. Bartlett, Sham M. Kakade +2

Existing theory suggests that for linear regression problems categorized by capacity and source conditions, gradient descent (GD) is always minimax optimal, while both ridge regres…