Escape dynamics and implicit bias of one-pass SGD in overparameterized quadratic networks

Dario Bocchi, Theotime Regimbeau, Carlo Lucibello +2

We analyze the one-pass stochastic gradient descent dynamics of a two-layer neural network with quadratic activations in a teacher--student framework. In the high-dimensional regim…