Stability and convergence analysis of AdaGrad for non-convex optimization via novel stopping time-based techniques

Ruinan Jin, Xiaoyu Wang, Baoxiang Wang

Adaptive gradient optimizers (AdaGrad), which dynamically adjust the learning rate based on iterative gradients, have emerged as powerful tools in deep learning. These adaptive met…