90 citations · 250 across the 45 of their papers we have counts for
3 papers · 1 filter
A Quantitative Survey of Communication Optimizations in Distributed Deep Learning
Shaohuai Shi, Zhenheng Tang, Xiaowen Chu +3
Nowadays, large and complex deep learning (DL) models are increasingly trained in a distributed manner across multiple worker machines, in which extensive communications between wo…
Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
Zhenheng Tang, Shaohuai Shi, Wei Wang +2
Distributed deep learning (DL) has become prevalent in recent years to reduce training time by leveraging multiple computing devices (e.g., GPUs/TPUs) due to larger models and data…
Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
Zhenheng Tang, Shaohuai Shi, Xiaowen Chu
Distributed learning techniques such as federated learning have enabled multiple workers to train machine learning models together to reduce the overall training time. However, cur…