1 citations · 1 across the 4 of their papers we have counts for
5 papers · 1 filter
Convergence Analysis of Nesterov's Accelerated Gradient Descent under Relaxed Assumptions
Chenhao Yu, Junhong Lin
We study convergence rates of Nesterov's Accelerated Gradient Descent (NAG) method for convex optimization in both deterministic and stochastic settings. We focus on a more general…
Convergence Analysis of Stochastic Accelerated Gradient Methods for Generalized Smooth Optimizations
Chenhao Yu, Yusu Hong, Junhong Lin
We investigate the Randomized Stochastic Accelerated Gradient (RSAG) method, utilizing either constant or adaptive step sizes, for stochastic optimization problems with generalized…
Revisiting Convergence of AdaGrad with Relaxed Assumptions
Yusu Hong, Junhong Lin
In this study, we revisit the convergence of AdaGrad with momentum (covering AdaGrad as a special case) on non-convex smooth optimization problems. We consider a general noise mode…
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions
Yusu Hong, Junhong Lin
The Adaptive Momentum Estimation (Adam) algorithm is highly effective in training various deep learning tasks. Despite this, there's limited theoretical understanding for Adam, esp…
High Probability Convergence of Adam Under Unbounded Gradients and Affine Variance Noise
Yusu Hong, Junhong Lin
In this paper, we study the convergence of the Adaptive Moment Estimation (Adam) algorithm under unconstrained non-convex smooth stochastic optimizations. Despite the widespread us…