176 citations · 265 across the 10 of their papers we have counts for
Showing cs.DCShow all
3 papers · 1 filter
cs.DC2018★ 3 cited
SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural Networks
Linnan Wang, Wei Wu, Junyu Zhang +4
The performance and efficiency of distributed training of Deep Neural Networks highly depend on the performance of gradient averaging among all participating nodes, which is bounde…
cs.DC2018★ 176 cited
SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks
Linnan Wang, Jinmian Ye, Yiyang Zhao +5
Going deeper and wider in neural architectures improves the accuracy, while the limited GPU DRAM places an undesired restriction on the network design domain. Deep Learning (DL) pr…
cs.DC2016★ 5 cited
Efficient Communications in Training Large Scale Neural Networks
Linnan Wang, Wei Wu, George Bosilca +2
We consider the problem of how to reduce the cost of communication that is required for the parallel training of a neural network. The state-of-the-art method, Bulk Synchronous Par…