Matrix-Free Least Squares Solvers: Values, Gradients, and What to Do With Them

Hrittik Roy, Søren Hauberg, Nicholas Krämer

This paper argues that the method of least squares has significant unfulfilled potential in modern machine learning, far beyond merely being a tool for fitting linear models. To re…