1 citations · 1 across the 7 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.CL2026
When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
Siyan Zhao, Yonggan Fu, Jindong Jiang +8
On-policy distillation (OPD) has become increasingly popular for transferring teacher capabilities to student models. In this work, we ask a critical research question: Is on-polic…
cs.LG2026
Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
Siyan Zhao, Zhihui Xie, Mengchen Liu +4
Knowledge distillation improves large language model (LLM) reasoning by compressing the knowledge of a teacher LLM to train smaller LLMs. On-policy distillation advances this appro…