Singular perturbations and hierarchical learning in two-layer neural networks

Cédric Gerbelot, Jean-Christophe Mourrat

We study the population gradient flow of an infinitely wide two-layer neural network learning a misspecified single-index model in high dimension. The two layers are optimized join…