5 citations · 10 across the 6 of their papers we have counts for
4 papers · 1 filter
Position Embedding Needs an Independent Layer Normalization
Runyi Yu, Zhennan Wang, Yinhuai Wang +5
The Position Embedding (PE) is critical for Vision Transformers (VTs) due to the permutation-invariance of self-attention operation. By analyzing the input and output of each encod…
ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentation
Kehan Li, Zhennan Wang, Zesen Cheng +6
Recently, self-supervised large-scale visual pre-training models have shown great promise in representing pixel-level semantic relationships, significantly promoting the developmen…
Locality Guidance for Improving Vision Transformers on Tiny Datasets
Kehan Li, Runyi Yu, Zhennan Wang +3
While the Vision Transformer (VT) architecture is becoming trendy in computer vision, pure VT models perform poorly on tiny datasets. To address this issue, this paper proposes the…
BN: Enhancing Batch Normalization by Equalizing the Norms of Features
Zhennan Wang, Kehan Li, Runyi Yu +7
In this paper, we analyze batch normalization from the perspective of discriminability and find the disadvantages ignored by previous studies: the difference in norms of samp…