1 paper
Meilin Yang, Zixuan Ding, Jianhao Nie +5
On-policy self-distillation (OPSD) adapts a language model by distilling guidance from a frozen teacher on trajectories sampled from the student. Its effectiveness, however, depend…