Asynchronous Training Schemes in Distributed Learning with Time Delay

Haoxiang Wang, Zhanhong Jiang, Chao Liu +3

In the context of distributed deep learning, the issue of stale weights or gradients could result in poor algorithmic performance. This issue is usually tackled by delay tolerant a…