95 citations · 430 across the 31 of their papers we have counts for
4 papers · 1 filter
Gradient Descent Can Take Exponential Time to Escape Saddle Points
Simon S. Du, Chi Jin, Jason D. Lee +3
Although gradient descent (GD) almost always escapes saddle points asymptotically [Lee et al., 2016], this paper shows that even with fairly natural random initialization schemes a…
Fast Stochastic Methods for Nonsmooth Nonconvex Optimization
Sashank J. Reddi, Suvrit Sra, Barnabas Poczos +1
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike…
Stochastic Variance Reduction for Nonconvex Optimization
Sashank J. Reddi, Ahmed Hefny, Suvrit Sra +2
We study nonconvex finite-sum problems and analyze stochastic variance reduced gradient (SVRG) methods for them. SVRG and related methods have recently surged into prominence for c…
Fast Incremental Method for Nonconvex Optimization
Sashank J. Reddi, Suvrit Sra, Barnabas Poczos +1
We analyze a fast incremental aggregated gradient method for optimizing nonconvex problems of the form . Specifically, we analyze the SAGA algorithm within an…