Stochastic Distributed Learning with Gradient Quantization and Variance Reduction

Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko +2

We consider distributed optimization where the objective function is spread among different devices, each sending incremental model updates to a central server. To alleviate the co…