Optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularization

Dmitry Kobak, Jonathan Lomond, Benoit Sanchez

A conventional wisdom in statistical learning is that large models require strong regularization to prevent overfitting. Here we show that this rule can be violated by linear regre…