Propagation of Chaos in One-hidden-layer Neural Networks beyond Logarithmic Time

Margalit Glasgow, Denny Wu, Joan Bruna

We study the approximation gap between the dynamics of a polynomial-width neural network and its infinite-width counterpart, both trained using projected gradient descent in the me…