Ridge Regularization: an Essential Concept in Data Science

Abstract

Ridge or more formally ${\ell }_2$ regularization shows up in many areas of statistics and machine learning. It is one of those essential devices that any good data scientist needs to master for their craft. In this brief ridge fest I have collected together some of the magic and beauty of ridge that my colleagues and I have encountered over the past 40 years in applied statistics.

17 pages, 5 figures. This paper was invited by Technometrics to appear in a special section to celebrate the 50th anniversary of the 1970 original ridge paper by Hoerl and Kennard