78 citations · 163 across the 5 of their papers we have counts for
Showing 2019Show all
2 papers · 1 filter
cs.DC2019
On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning
Aritra Dutta, El Houcine Bergou, Ahmed M. Abdelmoniem +4
Compressed communication, in the form of sparsification or quantization of stochastic gradients, is employed to reduce communication costs in distributed data-parallel training of…
cs.LG2019★ 66 cited
Natural Compression for Distributed Deep Learning
Samuel Horvath, Chen-Yu Ho, Ludovit Horvath +3
Modern deep learning models are often trained in parallel over a collection of distributed machines to reduce training time. In such settings, communication of model updates among…