1 paper · 1 filter
Chenxu Yang, Chuanyu Qin, Qingyi Si +7
On-policy distillation (OPD) has become a popular training paradigm in the LLM community. This paradigm selects a larger model as the teacher to provide dense, fine-grained signals…