Singular-limit analysis of gradient descent with noise injection

Anna Shalova, André Schlichting, Mark Peletier

We study the limiting dynamics of a large class of noisy gradient descent systems in the overparameterized regime. In this regime the set of global minimizers of the loss is large,…