5 citations · 10 across the 5 of their papers we have counts for
Showing 2018 · cs.LGShow all
2 papers · 2 filters
cs.LG2018
A Sufficient Condition for Convergences of Adam and RMSProp
Fangyu Zou, Li Shen, Zequn Jie +2
Adam and RMSProp are two of the most influential adaptive stochastic algorithms for training deep neural networks, which have been pointed out to be divergent even in the convex se…
cs.LG2018
A Unified Analysis of AdaGrad with Weighted Aggregation and Momentum Acceleration
Li Shen, Congliang Chen, Fangyu Zou +3
Integrating adaptive learning rate and momentum techniques into SGD leads to a large class of efficiently accelerated adaptive stochastic algorithms, such as AdaGrad, RMSProp, Adam…