The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study

Daniel S. Park, Jascha Sohl-Dickstein, Quoc V. Le +1

We investigate how the final parameters found by stochastic gradient descent are influenced by over-parameterization. We generate families of models by increasing the number of cha…