Dense Is Not Enough: Hierarchical Supervision Allocation for Long-Horizon On-Policy Distillation

Yuhao Sun, Binrui Wu, Zhuoer Xu +5

On-policy distillation (OPD) transfers the capabilities of a large language model to a smaller student by providing teacher supervision on the student's own rollouts. In long-horiz…