Memorize to Generalize: on the Necessity of Interpolation in High Dimensional Linear Regression

Chen Cheng, John Duchi, Rohith Kuditipudi

We examine the necessity of interpolation in overparameterized models, that is, when achieving optimal predictive risk in machine learning problems requires (nearly) interpolating…