1 citations · 1 across the 4 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
math.OC2024
Revisiting Convergence of AdaGrad with Relaxed Assumptions
Yusu Hong, Junhong Lin
In this study, we revisit the convergence of AdaGrad with momentum (covering AdaGrad as a special case) on non-convex smooth optimization problems. We consider a general noise mode…
math.OC2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions
Yusu Hong, Junhong Lin
The Adaptive Momentum Estimation (Adam) algorithm is highly effective in training various deep learning tasks. Despite this, there's limited theoretical understanding for Adam, esp…