Local Quadratic Convergence of Stochastic Gradient Descent with Adaptive Step Size

Adityanarayanan Radhakrishnan, Mikhail Belkin, Caroline Uhler

Establishing a fast rate of convergence for optimization methods is crucial to their applicability in practice. With the increasing popularity of deep learning over the past decade…