1 paper · 1 filter
Jingang Zhou, Yuyi Zhou, Haiyang Guo +6
On-policy distillation (OPD) is a promising approach for transferring knowledge between language models, where a student receives dense token-level supervision along its own genera…