LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs

Junlong Jia, Xing Wu, Chaochen Gao +8

High-quality long-context data is essential for training large language models (LLMs) capable of processing extensive documents, yet existing synthesis approaches using relevance-b…