78 citations · 163 across the 5 of their papers we have counts for
5 papers
Resource-Efficient Federated Learning
Ahmed M. Abdelmoniem, Atal Narayan Sahu, Marco Canini +1
Federated Learning (FL) enables distributed training by learners using local data, thereby enhancing privacy and reducing communication. However, it presents numerous challenges re…
Rethinking gradient sparsification as total error minimization
Atal Narayan Sahu, Aritra Dutta, Ahmed M. Abdelmoniem +3
Gradient compression is a widely-established remedy to tackle the communication bottleneck in distributed training of large deep neural networks (DNNs). Under the error-feedback fr…
On the Convergence Analysis of Asynchronous SGD for Solving Consistent Linear Systems
Atal Narayan Sahu, Aritra Dutta, Aashutosh Tiwari +1
In the realm of big data and machine learning, data-parallel, distributed stochastic algorithms have drawn significant attention in the present days.~While the synchronous versions…
On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning
Aritra Dutta, El Houcine Bergou, Ahmed M. Abdelmoniem +4
Compressed communication, in the form of sparsification or quantization of stochastic gradients, is employed to reduce communication costs in distributed data-parallel training of…
Natural Compression for Distributed Deep Learning
Samuel Horvath, Chen-Yu Ho, Ludovit Horvath +3
Modern deep learning models are often trained in parallel over a collection of distributed machines to reduce training time. In such settings, communication of model updates among…