Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process

Guy Blanc, Neha Gupta, Gregory Valiant +1

We consider networks, trained via stochastic gradient descent to minimize ${\ell }_2$ loss, with the training labels perturbed by independent noise at each iteration. We characterize…