Caramel: Accelerating Decentralized Distributed Deep Learning with Computation Scheduling

Sayed Hadi Hashemi, Sangeetha Abdu Jyothi, Brighten Godfrey +1

The method of choice for parameter aggregation in Deep Neural Network (DNN) training, a network-intensive task, is shifting from the Parameter Server model to decentralized aggrega…