Convergence of Large-Stepsize SGD for the Cross-entropy Loss via Stochastic Lyapunov Stability

Konstantinos Emmanouilidis, Lachlan MacDonald, Salma Tarmoun +1

Modern deep learning has been shown to operate using learning rates far larger than those justified by classical optimization theory. Most prior analyses of this phenomenon focus o…