4 papers · 1 filter
How to Attain Communication-Efficient DNN Training? Convert, Compress, Correct
Zhong-Jing Chen, Eduin E. Hernandez, Yu-Chih Huang +1
This paper introduces CO3 -- an algorithm for communication-efficient federated Deep Neural Network (DNN) training. CO3 takes its name from three processing applied which reduce th…
Convert, compress, correct: Three steps toward communication-efficient DNN training
Zhong-Jing Chen, Eduin E. Hernandez, Yu-Chih Huang +1
In this paper, we introduce a novel algorithm, , for communication-efficiency distributed Deep Neural Network (DNN) training. is a joint training/com…
DNN gradient lossless compression: Can GenNorm be the answer?
Zhong-Jing Chen, Eduin E. Hernandez, Yu-Chih Huang +1
In this paper, the problem of optimal gradient lossless compression in Deep Neural Network (DNN) training is considered. Gradient compression is relevant in many distributed DNN tr…
Speeding-Up Back-Propagation in DNN: Approximate Outer Product with Memory
Eduin E. Hernandez, Stefano Rini, Tolga M. Duman
In this paper, an algorithm for approximate evaluation of back-propagation in DNN training is considered, which we term Approximate Outer Product Gradient Descent with Memory (Mem-…