Asynchronous Stochastic Optimization Robust to Arbitrary Delays

Alon Cohen, Amit Daniely, Yoel Drori +2

We consider stochastic optimization with delayed gradients where, at each time step $t$, the algorithm makes an update using a stale stochastic gradient from step $t-d_t$ for som…