DeBiFormer: Vision Transformer with Deformable Agent Bi-level Routing Attention

Nguyen Huu Bao Long, Chenyu Zhang, Yuzhi Shi +4

Vision Transformers with various attention modules have demonstrated superior performance on vision tasks. While using sparsity-adaptive attention, such as in DAT, has yielded stro…