ELSA: Enhanced Local Self-Attention for Vision Transformer

Jingkai Zhou, Pichao Wang, Fan Wang +3

Self-attention is powerful in modeling long-range dependencies, but it is weak in local finer-level feature learning. The performance of local self-attention (LSA) is just on par w…