Adaptive Stochastic Gradient Descent for Fast and Communication-Efficient Distributed Learning

Serge Kas Hanna, Rawad Bitar, Parimal Parag +2

We consider the setting where a master wants to run a distributed stochastic gradient descent (SGD) algorithm on $n$ workers, each having a subset of the data. Distributed SGD may…