99 citations · 223 across the 15 of their papers we have counts for
4 papers · 1 filter
Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
Zhenheng Tang, Shaohuai Shi, Xiaowen Chu
Distributed learning techniques such as federated learning have enabled multiple workers to train machine learning models together to reduce the overall training time. However, cur…
Understanding Top-k Sparsification in Distributed Deep Learning
Shaohuai Shi, Xiaowen Chu, Ka Chun Cheung +1
Distributed stochastic gradient descent (SGD) algorithms are widely deployed in training large-scale deep learning models, while the communication overhead among workers becomes th…
Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
Shaohuai Shi, Zhenheng Tang, Qiang Wang +2
To reduce the long training time of large deep neural network (DNN) models, distributed synchronous stochastic gradient descent (S-SGD) is commonly used on a cluster of workers. Ho…
AutoML: A Survey of the State-of-the-Art
Xin He, Kaiyong Zhao, Xiaowen Chu
Deep learning (DL) techniques have penetrated all aspects of our lives and brought us great convenience. However, building a high-quality DL system for a specific task highly relie…