2 citations · 2 across the 10 of their papers we have counts for
1 paper · 1 filter
Justin Robert, Raheel Qader
On-policy distillation trains a language model on its own generations while a teacher scores them token by token. It combines the dense supervision of imitation learning with the o…