Nesterov's Accelerated Gradient and Momentum as approximations to Regularised Update Descent

Aleksandar Botev, Guy Lever, David Barber

We present a unifying framework for adapting the update direction in gradient-based iterative optimization methods. As natural special cases we re-derive classical momentum and Nes…