1 paper · 1 filter
Meilin Yang, Zixuan Ding, Jianhao Nie +5
On-policy self-distillation (OPSD) adapts a language model by distilling guidance from a frozen teacher on trajectories sampled from the student. Its effectiveness, however, depend…