Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation

Yuchen Cai, Ding Cao, Liang Lin +9

On-policy distillation (OPD) has emerged as an efficient post-training paradigm for large language models. However, existing studies largely attribute this advantage to denser and…