Exploring Fast and Communication-Efficient Algorithms in Large-scale Distributed Networks

Yue Yu, Jiaxiang Wu, Junzhou Huang

The communication overhead has become a significant bottleneck in data-parallel network with the increasing of model size and data samples. In this work, we propose a new algorithm…