Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK

Yuanzhi Li, Tengyu Ma, Hongyang R. Zhang

We consider the dynamic of gradient descent for learning a two-layer neural network. We assume the input $x\in {\mathbb{R}}^d$ is drawn from a Gaussian distribution and the label of $…