12 papers
WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-Training
Zehao Chen, Gongxun Li, Tianxiang Ai +9
On-policy distillation (OPD) aligns a student with a teacher on trajectories sampled from the student itself, reducing the train-test state mismatch of offline distillation. The sa…
Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs
Zixuan Huang, Yang Zhou, Kaixuan Wang +7
Recent Vision-Language-Action (VLA) models for autonomous driving (AD) increasingly utilize chain-of-thought (CoT) supervision to enhance the reasoning capabilities of their Vision…
Policy Improvement Reinforcement Learning
Huaiyang Wang, Xiaojie Li, Xiaohan Wang +10
Reinforcement learning has become a central post-training paradigm for improving LLM and agent capabilities. Yet existing RL post-training methods share a common blind spot: they c…
Heterogeneous Agent Collaborative Reinforcement Learning
Zhixia Zhang, Zixuan Huang, Gongxun Li +10
We introduce Heterogeneous Agent Collaborative Reinforcement Learning (HACRL), a new Reinforcement Learning from Verifiable Reward (RLVR) problem that addresses the inefficiencies…
Vinedresser3D: Agentic Text-guided 3D Editing
Yankuan Chi, Xiang Li, Zixuan Huang +1
Text-guided 3D editing aims to modify existing 3D assets using natural-language instructions. Current methods struggle to jointly understand complex prompts, automatically localize…
Federated Reasoning Distillation Framework with Model Learnability-Aware Data Allocation
Wei Guo, Siyuan Lu, Xiangdong Ran +8
Data allocation plays a critical role in federated large language model (LLM) and small language models (SLMs) reasoning collaboration. Nevertheless, existing data allocation metho…
UniARM: Towards a Unified Autoregressive Reward Model for Multi-Objective Test-Time Alignment
Hongyan Xie, Yikun Ban, Ruiyu Fang +6
Multi-objective alignment aims to align LLM responses with multiple human preference objectives. Among existing methods, guiding the generation of frozen LLMs through autoregressiv…
Does Your Reasoning Model Implicitly Know When to Stop Thinking?
Zixuan Huang, Xin Xia, Yuxi Ren +11
Recent advancements in large reasoning models (LRMs) have greatly improved their capabilities on complex reasoning tasks through Long Chains of Thought (CoTs). However, this approa…
Weak-Driven Learning: How Weak Agents make Strong Agents Stronger
Zehao Chen, Gongxun Li, Tianxiang Ai +9
As post-training optimization becomes central to improving large language models, we observe a persistent saturation bottleneck: once models grow highly confident, further training…
Real-Time Aligned Reward Model beyond Semantics
Zixuan Huang, Xin Xia, Yuxi Ren +10
Reinforcement Learning from Human Feedback (RLHF) is a pivotal technique for aligning large language models (LLMs) with human preferences, yet it is susceptible to reward overoptim…
Mitigating Spurious Correlations Between Question and Answer via Chain-of-Thought Correctness Perception Distillation
Hongyan Xie, Yitong Yao, Yikun Ban +6
Large language models (LLMs) excel at reasoning tasks but are expensive to deploy. Thus small language models (SLMs) are fine-tuned on CoT data generated by LLMs to copy LLMs' abil…
Adaptive Batch-Wise Sample Scheduling for Direct Preference Optimization
Zixuan Huang, Yikun Ban, Lean Fu +4
Direct Preference Optimization (DPO) has emerged as an effective approach for aligning large language models (LLMs) with human preferences. However, its performance is highly depen…