24 citations · 50 across the 5 of their papers we have counts for
10 papers · 1 filter
Bridging the Divide: Reconsidering Softmax and Linear Attention
Dongchen Han, Yifan Pu, Zhuofan Xia +6
Widely adopted in modern Vision Transformer designs, Softmax attention can effectively capture long-range visual information; however, it incurs excessive computational cost when d…
GSVA: Generalized Segmentation via Multimodal Large Language Models
Zhuofan Xia, Dongchen Han, Yizeng Han +3
Generalized Referring Expression Segmentation (GRES) extends the scope of classic RES to refer to multiple objects in one expression or identify the empty targets absent in the ima…
DAT++: Spatially Dynamic Vision Transformer with Deformable Attention
Zhuofan Xia, Xuran Pan, Shiji Song +2
Transformers have shown superior performance on various vision tasks. Their large receptive field endows Transformer models with higher representation power than their CNN counterp…
FLatten Transformer: Vision Transformer using Focused Linear Attention
Dongchen Han, Xuran Pan, Yizeng Han +2
The quadratic computation complexity of self-attention has been a persistent challenge when applying Transformer models to vision tasks. Linear attention, on the other hand, offers…
Dynamic Perceiver for Efficient Visual Recognition
Yizeng Han, Dongchen Han, Zeyu Liu +7
Early exiting has become a promising approach to improving the inference efficiency of deep networks. By structuring models with multiple classifiers (exits), predictions for ``eas…
Slide-Transformer: Hierarchical Vision Transformer with Local Self-Attention
Xuran Pan, Tianzhu Ye, Zhuofan Xia +2
Self-attention mechanism has been a key factor in the recent progress of Vision Transformer (ViT), which enables adaptive feature extraction from global contexts. However, existing…