3k citations · 24.2k across the 299 of their papers we have counts for
3 papers · 2 filters
Non-convex Finite-Sum Optimization Via SCSG Methods
Lihua Lei, Cheng Ju, Jianbo Chen +1
We develop a class of algorithms, as variants of the stochastically controlled stochastic gradient (SCSG) methods (Lei and Jordan, 2016), for the smooth non-convex finite-sum optim…
Gradient Descent Can Take Exponential Time to Escape Saddle Points
Simon S. Du, Chi Jin, Jason D. Lee +3
Although gradient descent (GD) almost always escapes saddle points asymptotically [Lee et al., 2016], this paper shows that even with fairly natural random initialization schemes a…
Breaking Locality Accelerates Block Gauss-Seidel
Stephen Tu, Shivaram Venkataraman, Ashia C. Wilson +3
Recent work by Nesterov and Stich showed that momentum can be used to accelerate the rate of convergence for block Gauss-Seidel in the setting where a fixed partitioning of the coo…