1 paper
Zixiang Ni, Zhuo Hu, Renjie Cao +8
On-policy distillation (OPD) trains a student on its own generated prefixes with token-level teacher feedback, but transmitting or storing the teacher's full-vocabulary distributio…