On the Convergence of AdaGrad(Norm) on ${\mathbb{R}}^d$: Beyond Convexity, Non-Asymptotic Rate and Acceleration

Zijian Liu, Ta Duy Nguyen, Alina Ene +1

Existing analysis of AdaGrad and other adaptive methods for smooth convex optimization is typically for functions with bounded domain diameter. In unconstrained problems, previous…