One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation

Justin Robert, Raheel Qader

On-policy distillation trains a language model on its own generations while a teacher scores them token by token. It combines the dense supervision of imitation learning with the o…