most citedFirst-order Methods Almost Always Avoid Saddle Points

76 citations · 76 across the 1 of their papers we have counts for

collaborators
Showing stat.MLShow all

2 papers · 1 filter

stat.ML2017★ 76 cited

First-order Methods Almost Always Avoid Saddle Points

Jason D. Lee, Ioannis Panageas, Georgios Piliouras +3

We establish that first-order methods avoid saddle points for almost all initializations. Our results apply to a wide variety of first-order methods, including gradient descent, bl…

stat.ML2016

Gradient Descent Converges to Minimizers

Jason D. Lee, Max Simchowitz, Michael I. Jordan +1

We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical system…