Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index models

Elizabeth Collins-Woodfin, Courtney Paquette, Elliot Paquette +1

We analyze the dynamics of streaming stochastic gradient descent (SGD) in the high-dimensional limit when applied to generalized linear models and multi-index models (e.g. logistic…