1 paper · 1 filter
Zhenxing Zhang, Jiayan Teng, Wenxu Wu +6
On-policy distillation (OPD) has demonstrated two important capabilities in language models: compressing large teachers into smaller students and merging expert models into a singl…