SMYRF: Efficient Attention using Asymmetric Clustering

Giannis Daras, Nikita Kitaev, Augustus Odena +1

We propose a novel type of balanced clustering algorithm to approximate attention. Attention complexity is reduced from $O(N^2)$ to $O(N\log N)$, where $N$ is the sequence length.…