Deep linear networks for regression are implicitly regularized towards flat minima

Pierre Marion, Lénaïc Chizat

The largest eigenvalue of the Hessian, or sharpness, of neural networks is a key quantity to understand their optimization dynamics. In this paper, we study the sharpness of deep l…