OPSDL: On-Policy Self-Distillation for Long-Context Language Models

Xinsen Zhang, Zhenkai Ding, Tianjun Pan +4

Extending the effective context length of large language models (LLMs) remains a central challenge for real-world applications. While recent post-training methods have made progres…