Homogenization of SGD in high-dimensions: Exact dynamics and generalization properties

Courtney Paquette, Elliot Paquette, Ben Adlam +1

We develop a stochastic differential equation, called homogenized SGD, for analyzing the dynamics of stochastic gradient descent (SGD) on a high-dimensional random least squares pr…