Gradient Norm Minimization of Nesterov Acceleration: $o(1/k^3)$

Shuo Chen, Bin Shi, Ya-xiang Yuan

In the history of first-order algorithms, Nesterov's accelerated gradient descent (NAG) is one of the milestones. However, the cause of the acceleration has been a mystery for a lo…