8 papers
Vision Transformers are Circulant Attention Learners
Dongchen Han, Tianyu Li, Ziyi Wang +1
The self-attention mechanism has been a key factor in the advancement of vision Transformers. However, its quadratic complexity imposes a heavy computational burden in high-resolut…
CauSight: Learning to Supersense for Visual Causal Discovery
Yize Zhang, Meiqi Chen, Sirui Chen +4
Causal thinking enables humans to understand not just what is seen, but why it happens. To replicate this capability in modern AI systems, we introduce the task of visual causal di…
ViT: Unlocking Test-Time Training in Vision
Dongchen Han, Yining Li, Tianyu Li +6
Test-Time Training (TTT) has recently emerged as a promising direction for efficient sequence modeling. TTT reformulates attention operation as an online learning problem, construc…
Lightweight Time Series Data Valuation on Time Series Foundation Models via In-Context Finetuning
Shunyu Wu, Tianyue Li, Yixuan Leng +4
Time series foundation models (TSFMs) have demonstrated increasing capabilities due to their extensive pretraining on large volumes of diverse time series data. Consequently, the q…
MoE-CE: Enhancing Generalization for Deep Learning based Channel Estimation via a Mixture-of-Experts Framework
Tianyu Li, Yan Xin, Jianzhong +1
Reliable channel estimation (CE) is fundamental for robust communication in dynamic wireless environments, where models must generalize across varying conditions such as signal-to-…
Secure Transfer Learning: Training Clean Models Against Backdoor in (Both) Pre-trained Encoders and Downstream Datasets
Yechao Zhang, Yuxuan Zhou, Tianyu Li +4
Transfer learning from pre-trained encoders has become essential in modern machine learning, enabling efficient model adaptation across diverse tasks. However, this combination of…