A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updates

Yossi Arjevani, Ohad Shamir, Nathan Srebro

We provide tight finite-time convergence bounds for gradient descent and stochastic gradient descent on quadratic functions, when the gradients are delayed and reflect iterates fro…