High-dimensional dynamics of generalization error in neural networks

Madhu S. Advani, Andrew M. Saxe

We perform an average case analysis of the generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant "high-dimensional" reg…