First-order methods almost always avoid saddle points: the case of vanishing step-sizes

Ioannis Panageas, Georgios Piliouras, Xiao Wang

In a series of papers \cite{LSJR16, PP17, LPP}, it was established that some of the most commonly used first order methods almost surely (under random initializations) and with ste…