117 citations · 196 across the 10 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2021★ 19 cited
Rethinking gradient sparsification as total error minimization
Atal Narayan Sahu, Aritra Dutta, Ahmed M. Abdelmoniem +3
Gradient compression is a widely-established remedy to tackle the communication bottleneck in distributed training of large deep neural networks (DNNs). Under the error-feedback fr…
cs.LG2021★ 6 cited
DeepReduce: A Sparse-tensor Communication Framework for Distributed Deep Learning
Kelly Kostopoulou, Hang Xu, Aritra Dutta +3
Sparse tensors appear frequently in distributed deep learning, either as a direct artifact of the deep neural network's gradients, or as a result of an explicit sparsification proc…