1 citations · 1 across the 20 of their papers we have counts for
10 papers · 1 filter
Teacher-Student Gaps Are Not Enough: Outcome-Guided On-Policy Distillation for Multi-Turn Autonomous Agents
Tong Zhang, Zhou Liu, Yihao Liu +8
On-policy distillation (OPD) trains a student on its own trajectories with dense teacher supervision. Recent work on OPD for multi-turn autonomous agents often treats large teacher…
Generative Query Suggestion via Intent Coverage and Query-Level Credit Assignment
Xinpeng Liu, Lu Ma, Jiayi Qiao +6
Generative query suggestion aims to enhance user engagement by anticipating user intents and recommending relevant follow-up queries. A central challenge is to generate slates whos…
SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation
Wen Wang, Jiahua Bao, Tu Yongsiqi +8
We aim to improve model performance in multi-reward reinforcement learning training process. Existing Group reward-Decoupled Normalization Policy Optimization (GDPO) has mitigated…
GDPO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization
Haotian Liu, Yihao Liu, Jingwei Ni +11
As LLMs advance, post-training reinforcement learning (RL) increasingly relies on multi-dimensional rewards to cultivate comprehensive capabilities. This shift demands new algorith…
Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill
Tao Chen, Gangwei Jiang, Pengyu Cheng +10
Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines. However, current rew…
CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVR
Sijia Cui, Pengyu Cheng, Jiajun Song +6
Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced the reasoning capacity of Large Language Models (LLMs). However, RLVR solely relies on final answer…
SiameseNorm: Breaking the Barrier to Reconciling Pre/Post-Norm
Tianyu Li, Dongchen Han, Zixuan Cao +7
The long-standing tension between Pre- and Post-Norm remains an open problem in Transformer architecture, reflecting a fundamental trade-off between training stability and represen…
Eliminating Inductive Bias in Reward Models with Information-Theoretic Guidance
Zhuo Li, Pengyu Cheng, Zhechao Yu +7
Reward models (RMs) are essential in reinforcement learning from human feedback (RLHF) to align large language models (LLMs) with human values. However, RM training data is commonl…
Bringing Stability to Diffusion: Decomposing and Reducing Variance of Training Masked Diffusion Models
Mengni Jia, Mengyu Zhou, Yihao Liu +2
Masked diffusion models (MDMs) are a promising alternative to autoregressive models (ARMs), but they suffer from inherently much higher training variance. High variance leads to no…
Search Self-play: Pushing the Frontier of Agent Capability without Supervision
Hongliang Lu, Yuhang Wen, Pengyu Cheng +7
Reinforcement learning with verifiable rewards (RLVR) has become the mainstream technique for training LLM agents. However, RLVR highly depends on well-crafted task queries and cor…