226 citations · 1.9k across the 128 of their papers we have counts for
6 papers · 2 filters
SGD with shuffling: optimal rates without component convexity and large epoch requirements
Kwangjun Ahn, Chulhee Yun, Suvrit Sra
We study without-replacement SGD for solving finite-sum optimization problems. Specifically, depending on how the indices of the finite-sum are shuffled, we consider the RandomShuf…
Beyond Worst-Case Analysis in Stochastic Approximation: Moment Estimation Improves Instance Complexity
Jingzhao Zhang, Hongzhou Lin, Subhro Das +2
We study oracle complexity of gradient based methods for stochastic approximation problems. Though in many settings optimal algorithms and tight lower bounds are known for such pro…
Understanding Nesterov's Acceleration via Proximal Point Method
Kwangjun Ahn, Suvrit Sra
The proximal point method (PPM) is a fundamental method in optimization that is often used as a building block for designing optimization algorithms. In this work, we use the PPM m…
On Tight Convergence Rates of Without-replacement SGD
Kwangjun Ahn, Suvrit Sra
For solving finite-sum optimization problems, SGD without replacement sampling is empirically shown to outperform SGD. Denoting by the number of components in the cost and …
Complexity of Finding Stationary Points of Nonsmooth Nonconvex Functions
Jingzhao Zhang, Hongzhou Lin, Stefanie Jegelka +2
We provide the first non-asymptotic analysis for finding stationary points of nonsmooth, nonconvex functions. In particular, we study the class of Hadamard semi-differentiable func…
From Nesterov's Estimate Sequence to Riemannian Acceleration
Kwangjun Ahn, Suvrit Sra
We propose the first global accelerated gradient method for Riemannian manifolds. Toward establishing our result we revisit Nesterov's estimate sequence technique and develop an al…