176 citations · 368 across the 18 of their papers we have counts for
5 papers · 1 filter
SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural Networks
Linnan Wang, Wei Wu, Junyu Zhang +4
The performance and efficiency of distributed training of Deep Neural Networks highly depend on the performance of gradient averaging among all participating nodes, which is bounde…
SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks
Linnan Wang, Jinmian Ye, Yiyang Zhao +5
Going deeper and wider in neural architectures improves the accuracy, while the limited GPU DRAM places an undesired restriction on the network design domain. Deep Learning (DL) pr…
Efficient Communications in Training Large Scale Neural Networks
Linnan Wang, Wei Wu, George Bosilca +2
We consider the problem of how to reduce the cost of communication that is required for the parallel training of a neural network. The state-of-the-art method, Bulk Synchronous Par…
Large Scale Artificial Neural Network Training Using Multi-GPUs
Linnan Wang, Wei Wu, Jianxiong Xiao +1
This paper describes a method for accelerating large scale Artificial Neural Networks (ANN) training using multi-GPUs by reducing the forward and backward passes to matrix multipli…
BLASX: A High Performance Level-3 BLAS Library for Heterogeneous Multi-GPU Computing
Linnan Wang, Wei Wu, Jianxiong Xiao +1
Basic Linear Algebra Subprograms (BLAS) are a set of low level linear algebra kernels widely adopted by applications involved with the deep learning and scientific computing. The m…