Variance-based Gradient Compression for Efficient Distributed Deep Learning

Yusuke Tsuzuku, Hiroto Imachi, Takuya Akiba

Due to the substantial computational cost, training state-of-the-art deep neural networks for large-scale datasets often requires distributed training using multiple computation wo…