3 papers
cs.CV2026
Distilling 3D Spatial Reasoning into a Lightweight Vision-Language Model with CoT
Alaa Asfour, Christopher Indris, Leihan Chen +2
Large-scale 3D vision-language models (VLMs) like LLaVA-3D offer strong spatial reasoning but are difficult to deploy due to high computational costs. We propose a knowledge distil…
cs.CV2025
Beyond Isolated Heads: Multi-Overlapped-Head Self-Attention for Vision Transformers
Tianxiao Zhang, Bo Luo, Guanghui Wang
Multi-Head Self-Attention (MHSA) is the cornerstone of Vision Transformers, allowing models to capture diverse feature representations by projecting tokens into independent subspac…
cs.CV2024
Depth-Wise Convolutions in Vision Transformers for Efficient Training on Small Datasets
Tianxiao Zhang, Wenju Xu, Bo Luo +1
The Vision Transformer (ViT) leverages the Transformer's encoder to capture global information by dividing images into patches and achieves superior performance across various comp…