2 papers
cs.CV2025
Beyond Isolated Heads: Multi-Overlapped-Head Self-Attention for Vision Transformers
Tianxiao Zhang, Bo Luo, Guanghui Wang
Multi-Head Self-Attention (MHSA) is the cornerstone of Vision Transformers, allowing models to capture diverse feature representations by projecting tokens into independent subspac…
cs.CV2024
Depth-Wise Convolutions in Vision Transformers for Efficient Training on Small Datasets
Tianxiao Zhang, Wenju Xu, Bo Luo +1
The Vision Transformer (ViT) leverages the Transformer's encoder to capture global information by dividing images into patches and achieves superior performance across various comp…