Rethinking Query-Key Pairwise Interactions in Vision Transformers

Cheng Li, Yangxin Liu

Vision Transformers have achieved state-of-the-art performance in many visual tasks. Due to the quadratic computational and memory complexities of self-attention, recent works eith…