Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex Optimization

Jun-Kun Wang, Jacob Abernethy

The Heavy Ball Method, proposed by Polyak over five decades ago, is a first-order method for optimizing continuous functions. While its stochastic counterpart has proven extremely…