22 citations · 64 across the 22 of their papers we have counts for
Showing 2018 · cs.LGShow all
2 papers · 2 filters
cs.LG2018
Towards Understanding Acceleration Tradeoff between Momentum and Asynchrony in Nonconvex Stochastic Optimization
Tianyi Liu, Shiyang Li, Jianping Shi +2
Asynchronous momentum stochastic gradient descent algorithms (Async-MSGD) is one of the most popular algorithms in distributed machine learning. However, its convergence properties…
cs.LG2018
A Diffusion Approximation Theory of Momentum SGD in Nonconvex Optimization
Tianyi Liu, Zhehui Chen, Enlu Zhou +1
Momentum Stochastic Gradient Descent (MSGD) algorithm has been widely applied to many nonconvex optimization problems in machine learning, e.g., training deep neural networks, vari…