15 citations · 49 across the 17 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.LG2022★ 6 cited
Adam Can Converge Without Any Modification On Update Rules
Yushun Zhang, Congliang Chen, Naichen Shi +2
Ever since Reddi et al. 2018 pointed out the divergence issue of Adam, many new variants have been designed to obtain convergence. However, vanilla Adam remains exceptionally popul…
cs.LG2022★ 3 cited
Efficient-Adam: Communication-Efficient Distributed Adam
Congliang Chen, Li Shen, Wei Liu +1
Distributed adaptive stochastic gradient methods have been widely used for large-scale nonconvex optimization, such as training deep learning models. However, their communication c…