67 citations · 69 across the 3 of their papers we have counts for
3 papers
cs.LG2021★ 1 cited
CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation
Enda Yu, Dezun Dong, Yemao Xu +2
Communication overhead is the key challenge for distributed training. Gradient compression is a widely used approach to reduce communication traffic. When combining with parallel c…
cs.LG2020★ 1 cited
OD-SGD: One-step Delay Stochastic Gradient Descent for Distributed Training
Yemao Xu, Dezun Dong, Weixia Xu +1
The training of modern deep learning neural network calls for large amounts of computation, which is often provided by GPUs or other specific accelerators. To scale out to achieve…
cs.DC2020★ 67 cited
Communication optimization strategies for distributed deep neural network training: A survey
Shuo Ouyang, Dezun Dong, Yemao Xu +1
Recent trends in high-performance computing and deep learning have led to the proliferation of studies on large-scale deep neural network training. However, the frequent communicat…