5 papers · 1 filter
C3: Rethinking Process Supervision via Confidence-Guided Credit Correction in Multi-Turn Reinforcement Learning
Ai Jian, Haoran Pan, Eryu Guo +5
Large language models are increasingly trained as multi-turn agents that interact with external tools. However, outcome-only reinforcement learning can produce a form of misguided…
Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders
Xin Li, Hao Jiang, Jinghao Guo +2
A recommendation request on Reddit draws several replies, each naming different titles. When an LLM recommender is trained on such threads with reinforcement learning (RL), which r…
Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation
Xin Li, Hao Jiang, Xin Gao +6
Reinforcement learning can turn one language model into several specialists, each excellent at a single skill such as mathematics, coding or following instructions, but users need…
Measuring Collapse and Correction in Homogeneous-Panel LLM Debate
Xin Li, Mengbing Liu, Chau Yuen
Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve, but movement is not necessarily improvement: the same discussion can rescue an in…
ErrOPD: When Does On-Policy Distillation Transfer Teacher Errors?
Xin Li, Hao Jiang, Annan Wang +2
A distilled student can copy its teacher's mistakes along with its skills. On-policy distillation (OPD), used in recent post-training recipes, has the teacher grade every token of…