Efficient Communications in Training Large Scale Neural Networks

Linnan Wang, Wei Wu, George Bosilca +2

We consider the problem of how to reduce the cost of communication that is required for the parallel training of a neural network. The state-of-the-art method, Bulk Synchronous Par…