High-Dimensional Analysis of Gradient Flow for Extensive-Width Quadratic Neural Networks

Simon Martin, Giulio Biroli, Francis Bach

We study the high-dimensional training dynamics of a shallow neural network with quadratic activation in a teacher-student setup. We focus on the extensive-width regime, where the…