Last iterate convergence of SGD for Least-Squares in the Interpolation regime

Aditya Varre, Loucas Pillaud-Vivien, Nicolas Flammarion

Motivated by the recent successes of neural networks that have the ability to fit the data perfectly and generalize well, we study the noiseless model in the fundamental least-squa…