67 citations · 163 across the 16 of their papers we have counts for
24 papers
Evaluation and Optimization of Gradient Compression for Distributed Deep Learning
Lin Zhang, Longteng Zhang, Shaohuai Shi +2
To accelerate distributed training, many gradient compression methods have been proposed to alleviate the communication bottleneck in synchronous stochastic gradient descent (S-SGD…
An Efficient Split Fine-tuning Framework for Edge and Cloud Collaborative Learning
Shaohuai Shi, Qing Yang, Yang Xiang +2
To enable the pre-trained models to be fine-tuned with local data on edge devices without sharing data with the cloud, we design an efficient split fine-tuning (SFT) framework for…
Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters
Yang Xiang, Zhihua Wu, Weibao Gong +15
The ever-growing model size and scale of compute have attracted increasing interests in training deep learning models over multiple nodes. However, when it comes to training on clo…
FADNet++: Real-Time and Accurate Disparity Estimation with Configurable Networks
Qiang Wang, Shaohuai Shi, Shizhen Zheng +2
Deep neural networks (DNNs) have achieved great success in the area of computer vision. The disparity estimation problem tends to be addressed by DNNs which achieve much better pre…
Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks
Shaohuai Shi, Lin Zhang, Bo Li
Distributed training with synchronous stochastic gradient descent (SGD) on GPU clusters has been widely used to accelerate the training process of deep models. However, SGD only ut…
Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans
Xin He, Shihao Wang, Xiaowen Chu +6
The COVID-19 pandemic has spread globally for several months. Because its transmissibility and high pathogenicity seriously threaten people's lives, it is crucial to accurately and…