3 citations · 3 across the 8 of their papers we have counts for
10 papers
SiameseNorm: Breaking the Barrier to Reconciling Pre/Post-Norm
Tianyu Li, Dongchen Han, Zixuan Cao +7
The long-standing tension between Pre- and Post-Norm remains an open problem in Transformer architecture, reflecting a fundamental trade-off between training stability and represen…
Vision Transformers are Circulant Attention Learners
Dongchen Han, Tianyu Li, Ziyi Wang +1
The self-attention mechanism has been a key factor in the advancement of vision Transformers. However, its quadratic complexity imposes a heavy computational burden in high-resolut…
CauSight: Learning to Supersense for Visual Causal Discovery
Yize Zhang, Meiqi Chen, Sirui Chen +4
Causal thinking enables humans to understand not just what is seen, but why it happens. To replicate this capability in modern AI systems, we introduce the task of visual causal di…
ViT: Unlocking Test-Time Training in Vision
Dongchen Han, Yining Li, Tianyu Li +6
Test-Time Training (TTT) has recently emerged as a promising direction for efficient sequence modeling. TTT reformulates attention operation as an online learning problem, construc…
Lightweight Time Series Data Valuation on Time Series Foundation Models via In-Context Finetuning
Shunyu Wu, Tianyue Li, Yixuan Leng +4
Time series foundation models (TSFMs) have demonstrated increasing capabilities due to their extensive pretraining on large volumes of diverse time series data. Consequently, the q…
MoE-CE: Enhancing Generalization for Deep Learning based Channel Estimation via a Mixture-of-Experts Framework
Tianyu Li, Yan Xin, Jianzhong +1
Reliable channel estimation (CE) is fundamental for robust communication in dynamic wireless environments, where models must generalize across varying conditions such as signal-to-…
Secure Transfer Learning: Training Clean Models Against Backdoor in (Both) Pre-trained Encoders and Downstream Datasets
Yechao Zhang, Yuxuan Zhou, Tianyu Li +4
Transfer learning from pre-trained encoders has become essential in modern machine learning, enabling efficient model adaptation across diverse tasks. However, this combination of…
OrchMLLM: Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training
Yijie Zheng, Bangjun Xiao, Lei Shi +7
Multimodal large language models (MLLMs), such as GPT-4o, are garnering significant attention. During the exploration of MLLM training, we identified Modality Composition Incoheren…
Disentangled Parameter-Efficient Linear Model for Long-Term Time Series Forecasting
Yuang Zhao, Tianyu Li, Jiadong Chen +3
Long-term Time Series Forecasting (LTSF) is crucial across various domains, but complex deep models like Transformers are often prone to overfitting on extended sequences. Linear F…
Hallucination Detection and Hallucination Mitigation: An Investigation
Junliang Luo, Tianyu Li, Di Wu +3
Large language models (LLMs), including ChatGPT, Bard, and Llama, have achieved remarkable successes over the last two years in a range of different applications. In spite of these…