Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Data

Xiaolu Wang, Yuchang Sun, Hoi-To Wai +1

We consider the distributed learning problem with data dispersed across multiple workers under the orchestration of a central server. Asynchronous Stochastic Gradient Descent (SGD)…