activity
20232026
most citedLinear attention is (maybe) all you need (to understand transformer optimization)

1 citations · 1 across the 5 of their papers we have counts for

collaborators
Showing 2024Show all

2 papers · 1 filter

cs.LG2024

Graph Transformers Dream of Electric Flow

Xiang Cheng, Lawrence Carin, Suvrit Sra

We show theoretically and empirically that the linear Transformer, when applied to graph data, can implement algorithms that solve canonical problems such as electric flow and eige…

math.OC2024

Riemannian Bilevel Optimization

Sanchayan Dutta, Xiang Cheng, Suvrit Sra

We develop new algorithms for Riemannian bilevel optimization. We focus in particular on batch and stochastic gradient-based methods, with the explicit goal of avoiding second-orde…