10 papers
WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-Training
Zehao Chen, Gongxun Li, Tianxiang Ai +9
On-policy distillation (OPD) aligns a student with a teacher on trajectories sampled from the student itself, reducing the train-test state mismatch of offline distillation. The sa…
Policy Improvement Reinforcement Learning
Huaiyang Wang, Xiaojie Li, Xiaohan Wang +10
Reinforcement learning has become a central post-training paradigm for improving LLM and agent capabilities. Yet existing RL post-training methods share a common blind spot: they c…
Multi-Objective Exploration and Preference Optimization via Mutual Information
Hongyan Xie, Yikun Ban, Ruiyu Fang +4
Aligning large language models with diverse and heterogeneous human values requires multi-objective alignment methods to effectively trade off conflicting preference dimensions. Cu…
Weak-Driven Learning: How Weak Agents make Strong Agents Stronger
Zehao Chen, Gongxun Li, Tianxiang Ai +9
As post-training optimization becomes central to improving large language models, we observe a persistent saturation bottleneck: once models grow highly confident, further training…
Does Your Reasoning Model Implicitly Know When to Stop Thinking?
Zixuan Huang, Xin Xia, Yuxi Ren +11
Recent advancements in large reasoning models (LRMs) have greatly improved their capabilities on complex reasoning tasks through Long Chains of Thought (CoTs). However, this approa…
Real-Time Aligned Reward Model beyond Semantics
Zixuan Huang, Xin Xia, Yuxi Ren +10
Reinforcement Learning from Human Feedback (RLHF) is a pivotal technique for aligning large language models (LLMs) with human preferences, yet it is susceptible to reward overoptim…