16 citations · 42 across the 9 of their papers we have counts for
4 papers · 1 filter
STL-SGD: Speeding Up Local SGD with Stagewise Communication Period
Shuheng Shen, Yifei Cheng, Jingchang Liu +1
Distributed parallel stochastic gradient descent algorithms are workhorses for large scale machine learning tasks. Among them, local stochastic gradient descent (Local SGD) has att…
Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation
Junliang Guo, Xu Tan, Linli Xu +3
Non-autoregressive translation (NAT) models remove the dependence on previous target tokens and generate all target tokens in parallel, resulting in significant inference speedup b…
Faster Distributed Deep Net Training: Computation and Communication Decoupled Stochastic Gradient Descent
Shuheng Shen, Linli Xu, Jingchang Liu +2
With the increase in the amount of data and the expansion of model scale, distributed parallel training becomes an important and successful technique to address the optimization ch…
Accelerating Stochastic Gradient Descent Using Antithetic Sampling
Jingchang Liu, Linli Xu
(Mini-batch) Stochastic Gradient Descent is a popular optimization method which has been applied to many machine learning applications. But a rather high variance introduced by the…