Flatness and Generalization: Learning Multi-Index Models with Homogeneous Neural Networks

Harsh Vardhan, Hossein Taheri, Arya Mazumdar

A common heuristic used to explain the generalization of first-order gradient methods on non-convex neural networks is that "flat interpolators generalize well" (Hochreiter and Sch…