PolicyLong: Towards On-Policy Context Extension

Junlong Jia, Ziyang Chen, Xing Wu +4

Extending LLM context windows is hindered by scarce high-quality long-context data. Recent methods synthesize data with genuine long-range dependencies via information-theoretic ve…