Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions

Bohan Wang, Huishuai Zhang, Zhi-Ming Ma +1

We provide a simple convergence proof for AdaGrad optimizing non-convex objectives under only affine noise variance and bounded smoothness assumptions. The proof is essentially bas…