Global Exponential Convergence of Two-Layer Linear Network Training

Stephen Y Zhang, Gabriel Peyré

We prove global exponential (linear) convergence with an explicit rate in the rich scaling for wide two-layer linear networks trained with smooth Polyak-Lojasiewicz predictor losse…