8 papers
Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation
Jie Sun, Mao Zheng, Mingyang Song +8
Multi-teacher on-policy distillation allows a student to learn from complementary specialists on its own trajectories. Domain-routed approaches, however, select one teacher per exa…
DASH: Divergence-Adaptive Supervision Horizons for On-Policy Self-Distillation of Reasoning Models
ZhiYan Hou, Xinyu Tang, Hongyan An +9
Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models using automatically verifiable outcome signals, but these signals…
EasyOPD: An Easy-to-use On-Policy Distillation Framework for Large Language Models
Jie Sun, Mao Zheng, Mingyang Song +7
Conventional language-model distillation often relies on fixed teacher-generated data, which may not cover the states encountered by an evolving student policy. On-policy distillat…
On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents
Gengsheng Li, Mao Zheng, Mingyang Song +8
Multi-turn agents that plan, invoke tools, and interact with environments offer a promising paradigm for solving complex tasks, yet their capabilities typically rely on very large…
Visual-Advantage On-Policy Distillation for Vision-Language Models
Ruiqi Liu, Xiaolei Lv, Gengsheng Li +8
On-policy knowledge distillation has proven effective for language models, yet its application to vision-language models (VLMs) remains underexplored. We observe that standard on-p…
Rubric-based On-policy Distillation
Junfeng Fang, Zhepei Hong, Mao Zheng +7
On-policy distillation (OPD) is a powerful paradigm for model alignment, yet its reliance on teacher logits restricts its application to white-box scenarios. We contend that struct…
Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
Gengsheng Li, Tianyu Yang, Junfeng Fang +6
Reinforcement learning with verifiable rewards (RLVR) has become a standard paradigm for post-training large language models. While Group Relative Policy Optimization (GRPO) is wid…
R-Diverse: Mitigating Diversity Illusion in Self-Play LLM Training
Gengsheng Li, Jinghan He, Shijie Wang +7
Self-play bootstraps LLM reasoning through an iterative Challenger-Solver loop: the Challenger is trained to generate questions that target the Solver's capabilities, and the Solve…