WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-Training

Zehao Chen, Gongxun Li, Tianxiang Ai +9

On-policy distillation (OPD) aligns a student with a teacher on trajectories sampled from the student itself, reducing the train-test state mismatch of offline distillation. The sa…