1 paper
Zhengyu Fang, Seoyeon Hong, Jie Yang +4
On-policy distillation (OPD) trains a student on the responses it generates. Existing LLM multi-teacher OPD transfers what specialists predict through their output distributions. W…