Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks

Benjamin Bowman, Guido Montufar

We study the dynamics of a neural network in function space when optimizing the mean squared error via gradient flow. We show that in the underparameterized regime the network lear…