226 citations · 1.9k across the 128 of their papers we have counts for
6 papers · 1 filter
Randomized Block Coordinate DC Programming
Hoomaan Maskan, Paniz Halvachi, Suvrit Sra +1
We introduce an extension of the Difference of Convex Algorithm (DCA) in the form of a randomized block coordinate approach for problems with separable structure. For coordinat…
Graph Transformers Dream of Electric Flow
Xiang Cheng, Lawrence Carin, Suvrit Sra
We show theoretically and empirically that the linear Transformer, when applied to graph data, can implement algorithms that solve canonical problems such as electric flow and eige…
Toward generalizable learning of all (linear) first-order methods via memory augmented Transformers
Sanchayan Dutta, Suvrit Sra
We show that memory-augmented Transformers can implement the entire class of linear first-order methods (LFOMs), a class that contains gradient descent (GD) and more advanced metho…
First-Order Methods for Linearly Constrained Bilevel Optimization
Guy Kornowski, Swati Padmanabhan, Kai Wang +2
Algorithms for bilevel optimization often encounter Hessian computations, which are prohibitive in high dimensions. While recent works offer first-order methods for unconstrained b…
Riemannian Bilevel Optimization
Sanchayan Dutta, Xiang Cheng, Suvrit Sra
We develop new algorithms for Riemannian bilevel optimization. We focus in particular on batch and stochastic gradient-based methods, with the explicit goal of avoiding second-orde…
Efficient Sampling on Riemannian Manifolds via Langevin MCMC
Xiang Cheng, Jingzhao Zhang, Suvrit Sra
We study the task of efficiently sampling from a Gibbs distribution over a Riemannian manifold via (geometric) Langevin MCMC; this algorithm involves…