FLuRKA: Fast and accurate unified Low-Rank & Kernel Attention

Ahan Gupta, Hao Guo, Yueming Yuan +2

Many efficient $\text{approximate}$ self-attention techniques have become prevalent since the inception of the transformer architecture. Two popular classes of these techniques a…