13 citations · 35 across the 13 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2021★ 13 cited
ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training
Chia-Yu Chen, Jiamin Ni, Songtao Lu +8
Large-scale distributed training of Deep Neural Networks (DNNs) on state-of-the-art platforms is expected to be severely communication constrained. To overcome this limitation, num…
cs.LG2020★ 8 cited
A(DP)SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy
Jie Xu, Wei Zhang, Fei Wang
As deep learning models are usually massive and complex, distributed learning is essential for increasing training efficiency. Moreover, in many real-world application scenarios li…
cs.LG2020★ 1 cited
Improving Efficiency in Large-Scale Decentralized Distributed Training
Wei Zhang, Xiaodong Cui, Abdullah Kayi +9
Decentralized Parallel SGD (D-PSGD) and its asynchronous variant Asynchronous Parallel SGD (AD-PSGD) is a family of distributed learning algorithms that have been demonstrated to p…