Non-Euclidean Gradient Descent Has Many Edges of Stability

Peter Zaika, Zakhar Shumaylov

Recent observations that deep learning models train at the edge of stability have renewed interest in understanding stability properties of classical optimization algorithms. For q…