Showing math.OCShow all
3 papers · 1 filter
math.OC2025
Convergence Analysis of Stochastic Accelerated Gradient Methods for Generalized Smooth Optimizations
Chenhao Yu, Yusu Hong, Junhong Lin
We investigate the Randomized Stochastic Accelerated Gradient (RSAG) method, utilizing either constant or adaptive step sizes, for stochastic optimization problems with generalized…
math.OC2025
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions
Yusu Hong, Junhong Lin
The Adaptive Momentum Estimation (Adam) algorithm is highly effective in training various deep learning tasks. Despite this, there's limited theoretical understanding for Adam, esp…
math.OC2024
Revisiting Convergence of AdaGrad with Relaxed Assumptions
Yusu Hong, Junhong Lin
In this study, we revisit the convergence of AdaGrad with momentum (covering AdaGrad as a special case) on non-convex smooth optimization problems. We consider a general noise mode…