24 citations · 24 across the 2 of their papers we have counts for
4 papers
Step by Step Network
Dongchen Han, Tianzhu Ye, Zhuofan Xia +4
Scaling up network depth is a fundamental pursuit in neural architecture design, as theory suggests that deeper models offer exponentially greater capability. Benefiting from the r…
Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials
Yifan Pu, Jixuan Ying, Qixiu Li +7
Vision Transformers (ViTs) have become a universal backbone for both image recognition and image generation. Yet their Multi-Head Self-Attention (MHSA) layer still performs a quadr…
Differential Transformer
Tianzhu Ye, Li Dong, Yuqing Xia +4
Transformer tends to overallocate attention to irrelevant context. In this work, we introduce Diff Transformer, which amplifies attention to the relevant context while canceling no…
Contrastive Language-Image Pre-Training with Knowledge Graphs
Xuran Pan, Tianzhu Ye, Dongchen Han +2
Recent years have witnessed the fast development of large-scale pre-training frameworks that can extract multi-modal representations in a unified form and achieve promising perform…