Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Revealing the Learning Dynamics of Long-Context Continual Pre-training
Yupu Liang, Shuang Chen, Guanwei Zhang +2
Existing studies on Long-Context Continual Pre-training (LCCP) mainly focus on small-scale models and limited data regimes (tens of billions of tokens). We argue that directly migr…
cs.CL2025
Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts
Yifei Yu, Qian-Wen Zhang, Lingfeng Qiao +7
Evaluating the ability of large language models (LLMs) to process lengthy contexts is critical, especially for retrieving query-relevant information embedded within them. We introd…