Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stability

Tongtong Liang, Esha Singh, Rahul Parhi +2

Gradient descent on overparameterized neural networks typically operates at the Edge of Stability (EoS), where the largest Hessian eigenvalue hovers around a step-size-dependent th…