STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization

Feng Zhu, Jingjing Zhang, Xin Wang

Synchronous local stochastic gradient descent (local SGD) suffers from some workers being idle and random delays due to slow and straggling workers, as it waits for the workers to…