226 citations · 1.9k across the 128 of their papers we have counts for
6 papers · 2 filters
R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate
Jingzhao Zhang, Hongyi Zhang, Suvrit Sra
We study smooth stochastic optimization problems on Riemannian manifolds. Via adapting the recently proposed SPIDER algorithm \citep{fang2018spider} (a variance reduced stochastic…
Efficiently testing local optimality and escaping saddles for ReLU networks
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
We provide a theoretical algorithm for checking local optimality and escaping saddles at nondifferentiable points of empirical risks of two-layer ReLU networks. Our algorithm recei…
Towards Riemannian Accelerated Gradient Methods
Hongyi Zhang, Suvrit Sra
We propose a Riemannian version of Nesterov's Accelerated Gradient algorithm (RAGD), and show that for geodesically smooth and strongly convex problems, within a neighborhood of th…
Random Shuffling Beats SGD after Finite Epochs
Jeff Z. HaoChen, Suvrit Sra
A long-standing problem in the theory of stochastic gradient descent (SGD) is to prove that its without-replacement version RandomShuffle converges faster than the usual with-repla…
Direct Runge-Kutta Discretization Achieves Acceleration
Jingzhao Zhang, Aryan Mokhtari, Suvrit Sra +1
We study gradient-based optimization methods obtained by directly discretizing a second-order ordinary differential equation (ODE) related to the continuous limit of Nesterov's acc…
New concavity and convexity results for symmetric polynomials and their ratios
Suvrit Sra
We prove some "power" generalizations of Marcus-Lopes-style (including McLeod and Bullen) concavity inequalities for elementary symmetric polynomials, and convexity inequalities (o…