2 papers
cs.CL2025
EntropyLong: Effective Long-Context Training via Predictive Uncertainty
Junlong Jia, Ziyang Chen, Xing Wu +5
Training long-context language models to capture long-range dependencies requires specialized data construction. Current approaches, such as generic text concatenation or heuristic…
cs.CL2025
LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs
Junlong Jia, Xing Wu, Chaochen Gao +8
High-quality long-context data is essential for training large language models (LLMs) capable of processing extensive documents, yet existing synthesis approaches using relevance-b…