51 citations · 91 across the 8 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.CV2024
Beyond Isolated Heads: Multi-Overlapped-Head Self-Attention for Vision Transformers
Tianxiao Zhang, Bo Luo, Guanghui Wang
Multi-Head Self-Attention (MHSA) is the cornerstone of Vision Transformers, allowing models to capture diverse feature representations by projecting tokens into independent subspac…
cs.CV2024★ 51 cited
Depth-Wise Convolutions in Vision Transformers for Efficient Training on Small Datasets
Tianxiao Zhang, Wenju Xu, Bo Luo +1
The Vision Transformer (ViT) leverages the Transformer's encoder to capture global information by dividing images into patches and achieves superior performance across various comp…