Optimization Insights into Deep Diagonal Linear Networks

Hippolyte Labarrière, Cesare Molinari, Lorenzo Rosasco +2

Gradient-based methods successfully train highly overparameterized models in practice, even though the associated optimization problems are markedly nonconvex. Understanding the me…