Conflicting Biases at the Edge of Stability: Norm versus Sharpness Regularization

Maria Matveev, Vit Fojtik, Hung-Hsu Chou +2

The remarkable generalization properties of overparameterized networks are often attributed to implicit biases, such as norm minimization at small learning rates and low sharpness…