EasyOPD: An Easy-to-use On-Policy Distillation Framework for Large Language Models

Jie Sun, Mao Zheng, Mingyang Song +7

Conventional language-model distillation often relies on fixed teacher-generated data, which may not cover the states encountered by an evolving student policy. On-policy distillat…