5 citations · 6 across the 10 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.LG2024
A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD
Ruinan Jin, Xiao Li, Yaoliang Yu +1
Adaptive Moment Estimation (Adam) is a cornerstone optimization algorithm in deep learning, widely recognized for its flexibility with adaptive learning rates and efficiency in han…
math.OC2024
A KL-based Analysis Framework with Applications to Non-Descent Optimization Methods
Junwen Qiu, Bohao Ma, Xiao Li +1
We propose a novel analysis framework for non-descent-type optimization methodologies in nonconvex scenarios based on the Kurdyka-Lojasiewicz property. Our framework allows coverin…
math.OC2024
A Generalized Version of Chung's Lemma and its Applications
Li Jiang, Xiao Li, Andre Milzarek +1
Chung's Lemma is a classical tool for establishing asymptotic convergence rates of (stochastic) optimization methods under strong convexity-type assumptions and appropriate polynom…