Robustifying Asynchronous SGD via Soft Throttling

Kaoru Otsuka, Maxime Meyer, Yuki Takezawa +2

Asynchronous SGD is a popular algorithm for distributed learning where each client's gradient update is applied on arrival. This leads to a speed-up, but also an increased vulnerab…