From Averaging to Acceleration, There is Only a Step-size

Nicolas Flammarion, Francis Bach

We show that accelerated gradient descent, averaged gradient descent and the heavy-ball method for non-strongly-convex problems may be reformulated as constant parameter second-ord…