On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization

Dongruo Zhou, Jinghui Chen, Yuan Cao +2

Adaptive gradient methods are workhorses in deep learning. However, the convergence guarantees of adaptive gradient methods for nonconvex optimization have not been thoroughly stud…