1 paper · 1 filter
Wei Fan, Qingyue Yang, Chao-Hong Tan +6
On-policy self-distillation improves a model using guidance from the model itself, but it still requires a teacher that provides more useful predictions than the student. Existing…