226 citations · 1.9k across the 128 of their papers we have counts for
11 papers · 1 filter
R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate
Jingzhao Zhang, Hongyi Zhang, Suvrit Sra
We study smooth stochastic optimization problems on Riemannian manifolds. Via adapting the recently proposed SPIDER algorithm \citep{fang2018spider} (a variance reduced stochastic…
Deep-RBF Networks Revisited: Robust Classification with Rejection
Pourya Habib Zadeh, Reshad Hosseini, Suvrit Sra
One of the main drawbacks of deep neural networks, like many other classifiers, is their vulnerability to adversarial attacks. An important reason for their vulnerability is assign…
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
We study finite sample expressivity, i.e., memorization power of ReLU networks. Recent results require hidden nodes to memorize/interpolate arbitrary data points. In contra…
Efficiently testing local optimality and escaping saddles for ReLU networks
Chulhee Yun, Suvrit Sra, Ali Jadbabaie
We provide a theoretical algorithm for checking local optimality and escaping saddles at nondifferentiable points of empirical risks of two-layer ReLU networks. Our algorithm recei…
Towards Riemannian Accelerated Gradient Methods
Hongyi Zhang, Suvrit Sra
We propose a Riemannian version of Nesterov's Accelerated Gradient algorithm (RAGD), and show that for geodesically smooth and strongly convex problems, within a neighborhood of th…
Random Shuffling Beats SGD after Finite Epochs
Jeff Z. HaoChen, Suvrit Sra
A long-standing problem in the theory of stochastic gradient descent (SGD) is to prove that its without-replacement version RandomShuffle converges faster than the usual with-repla…