2 papers
cs.CL2026
Revealing the Learning Dynamics of Long-Context Continual Pre-training
Yupu Liang, Shuang Chen, Guanwei Zhang +2
Existing studies on Long-Context Continual Pre-training (LCCP) mainly focus on small-scale models and limited data regimes (tens of billions of tokens). We argue that directly migr…
cs.CL2026
CL-bench: A Benchmark for Context Learning
Shihan Dou, Ming Zhang, Zhangyue Yin +24
Current language models (LMs) excel at reasoning over prompts using pre-trained knowledge. However, real-world tasks are far more complex and context-dependent: models must learn f…