DBA: Efficient Transformer with Dynamic Bilinear Low-Rank Attention

Bosheng Qin, Juncheng Li, Siliang Tang +1

Many studies have been conducted to improve the efficiency of Transformer from quadric to linear. Among them, the low-rank-based methods aim to learn the projection matrices to com…