256 citations · 627 across the 29 of their papers we have counts for
4 papers · 1 filter
Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization
Tian Ye, Simon S. Du
We study the asymmetric low-rank factorization problem: \[\min_{\mathbf{U} \in \mathbb{R}^{m \times d}, \mathbf{V} \in \mathbb{R}^{n \times d}} \frac{1}{2}\|\mathbf{U}\mathbf{V}^\t…
Acceleration via Symplectic Discretization of High-Resolution Differential Equations
Bin Shi, Simon S. Du, Weijie J. Su +1
We study first-order optimization methods obtained by discretizing ordinary differential equations (ODEs) corresponding to Nesterov's accelerated gradient methods (NAGs) and Polyak…
Linear Convergence of the Primal-Dual Gradient Method for Convex-Concave Saddle Point Problems without Strong Convexity
Simon S. Du, Wei Hu
We consider the convex-concave saddle point problem where is smooth and convex and is smooth and strongly convex. We prove that if t…
Gradient Descent Can Take Exponential Time to Escape Saddle Points
Simon S. Du, Chi Jin, Jason D. Lee +3
Although gradient descent (GD) almost always escapes saddle points asymptotically [Lee et al., 2016], this paper shows that even with fairly natural random initialization schemes a…