Learning Under Delayed Feedback: Implicitly Adapting to Gradient Delays

Rotem Zamir Aviv, Ido Hakimi, Assaf Schuster +1

We consider stochastic convex optimization problems, where several machines act asynchronously in parallel while sharing a common memory. We propose a robust training method for th…