10 papers
SiameseNorm: Breaking the Barrier to Reconciling Pre/Post-Norm
Tianyu Li, Dongchen Han, Zixuan Cao +7
The long-standing tension between Pre- and Post-Norm remains an open problem in Transformer architecture, reflecting a fundamental trade-off between training stability and represen…
Geo-R1: Improving Few-Shot Geospatial Referring Expression Understanding with Reinforcement Fine-Tuning
Zilun Zhang, Zian Guan, Tiancheng Zhao +7
Referring expression understanding in remote sensing poses unique challenges, as it requires reasoning over complex object-context relationships. While supervised fine-tuning (SFT)…
ViT: Unlocking Test-Time Training in Vision
Dongchen Han, Yining Li, Tianyu Li +6
Test-Time Training (TTT) has recently emerged as a promising direction for efficient sequence modeling. TTT reformulates attention operation as an online learning problem, construc…
OrchMLLM: Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training
Yijie Zheng, Bangjun Xiao, Lei Shi +7
Multimodal large language models (MLLMs), such as GPT-4o, are garnering significant attention. During the exploration of MLLM training, we identified Modality Composition Incoheren…
Lightweight Time Series Data Valuation on Time Series Foundation Models via In-Context Finetuning
Shunyu Wu, Tianyue Li, Yixuan Leng +4
Time series foundation models (TSFMs) have demonstrated increasing capabilities due to their extensive pretraining on large volumes of diverse time series data. Consequently, the q…
Disentangled Parameter-Efficient Linear Model for Long-Term Time Series Forecasting
Yuang Zhao, Tianyu Li, Jiadong Chen +3
Long-term Time Series Forecasting (LTSF) is crucial across various domains, but complex deep models like Transformers are often prone to overfitting on extended sequences. Linear F…