A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks

Zhengdao Chen, Eric Vanden-Eijnden, Joan Bruna

To understand the training dynamics of neural networks, prior studies have considered the mean-field limit of two-layer neural networks as the width tends to infinity, establishing…