136 citations · 436 across the 9 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2023★ 4 cited
UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization
Yiming Jiang, Jinlan Liu, Dongpo Xu +1
Adam-type algorithms have become a preferred choice for optimisation in the deep learning setting, however, despite success, their convergence is still not well understood. To this…
cs.LG2021
Scaling transition from momentum stochastic gradient descent to plain stochastic gradient descent
Kun Zeng, Jinlan Liu, Zhixia Jiang +1
The plain stochastic gradient descent and momentum stochastic gradient descent have extremely wide applications in deep learning due to their simple settings and low computational…
cs.LG2021★ 2 cited
A decreasing scaling transition scheme from Adam to SGD
Kun Zeng, Jinlan Liu, Zhixia Jiang +1
Adaptive gradient algorithm (AdaGrad) and its variants, such as RMSProp, Adam, AMSGrad, etc, have been widely used in deep learning. Although these algorithms are faster in the ear…