3 papers
cs.CV2025
Step by Step Network
Dongchen Han, Tianzhu Ye, Zhuofan Xia +4
Scaling up network depth is a fundamental pursuit in neural architecture design, as theory suggests that deeper models offer exponentially greater capability. Benefiting from the r…
cs.CV2025
Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials
Yifan Pu, Jixuan Ying, Qixiu Li +7
Vision Transformers (ViTs) have become a universal backbone for both image recognition and image generation. Yet their Multi-Head Self-Attention (MHSA) layer still performs a quadr…
cs.CL2025
Differential Transformer
Tianzhu Ye, Li Dong, Yuqing Xia +4
Transformer tends to overallocate attention to irrelevant context. In this work, we introduce Diff Transformer, which amplifies attention to the relevant context while canceling no…