First-order Methods Almost Always Avoid Saddle Points

Jason D. Lee, Ioannis Panageas, Georgios Piliouras +3

We establish that first-order methods avoid saddle points for almost all initializations. Our results apply to a wide variety of first-order methods, including gradient descent, bl…