On the Position Bias of On-Policy Distillation

Yan Xie, Sijie Zhu, Tiansheng Wen +2

On-Policy Distillation (OPD) improves the learning efficiency of standard reinforcement learning through dense, token-level supervision from teachers. In the standard KL objective…