1 paper · 1 filter
Yingdong Shi, Zhihong Li, Changming Li +3
On-policy distillation (OPD) improves language-model by supervising a student on prefixes sampled from its own policy, but its supervision remains confined to the output space. Con…