A Geometric Structure of Acceleration and Its Role in Making Gradients Small Fast

Jongmin Lee, Chanwoo Park, Ernest K. Ryu

Since Nesterov's seminal 1983 work, many accelerated first-order optimization methods have been proposed, but their analyses lacks a common unifying structure. In this work, we ide…