3 papers
math.OC2026
Asymptotic Convergence and Stability of Adaptive Gradient Methods in Smooth Non-convex Optimization
Ruinan Jin, Xiaoyu Wang
Adaptive gradient methods, such as AdaGrad, have become fundamental tools in deep learning. Despite their widespread use, the asymptotic convergence of AdaGrad remains poorly under…
cs.LG2025
A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD
Ruinan Jin, Xiao Li, Yaoliang Yu +1
Adaptive Moment Estimation (Adam) is a cornerstone optimization algorithm in deep learning, widely recognized for its flexibility with adaptive learning rates and efficiency in han…
math.OC2024
Stability and convergence analysis of AdaGrad for non-convex optimization via novel stopping time-based techniques
Ruinan Jin, Xiaoyu Wang, Baoxiang Wang
Adaptive gradient optimizers (AdaGrad), which dynamically adjust the learning rate based on iterative gradients, have emerged as powerful tools in deep learning. These adaptive met…