1 paper · 1 filter
Bingxi Hou, Guochao Jiang, Guofeng Quan +4
On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student predictions requires alignment…