12 papers
Uncovering and Mitigating Aggregation-Induced Reward Hacking in Multi-Reward Reinforcement Learning
Yu Yuan, Yaoyou Fan, Lili Zhao +5
Reinforcement learning fine-tuning of large language models increasingly adopts multiple reward dimensions, including verifiable rules, task-specific evaluators, and learned reward…
Retrieval Grounding Latent Reasoning for Dense Retrieval
Gang Zhou, Xiongxi Yu, Hu Tian +5
Reasoning-intensive retrieval requires text representations to capture not only semantic similarity, but also the reasoning needed to determine relevance under a given retrieval in…
Long SKILL Compliance as Logical Reasoning: Closure-Grounded Detection with Scaling-Guided On-Policy Distillation
Shuaitao Zhao, Feng Ni, Lichao Ma +4
The increasing complexity of enterprise business scenarios has promoted the widespread adoption of long SKILL documents in agent systems, posing new challenges for compliance detec…
How Much, Then Where: Credit-Conserving Action-to-Token Allocation for Multi-Turn Agent Reinforcement Learning
Lichao Ma, Yang Sun, Shuaitao Zhao +9
Credit assignment in multi-turn agent reinforcement learning operates at two levels: assigning trajectory-level credit to actions and distributing each action's credit across its t…
Agentic Reinforcement Learning with Observation-Calibrated Self-Distillation
Yi Yang, Cong Qin, Xiaodan Liu +8
Large language model agents are commonly trained through reinforcement learning with sparse trajectory-level rewards, which offer limited guidance on how strongly individual tokens…
Look Ahead Before You Distill: Future Trajectory Validation of Teacher Guidance for Agentic On-Policy Distillation
Chishui Chen, Yaoyou Fan, Te Sun +11
On-policy distillation (OPD) provides teacher supervision on states visited by the student, reducing the distribution gap between training and inference. However, in multi-turn age…
Rethinking Continual Experience Internalization for Self-Evolving LLM Agents
Jingwen Chen, Wenkai Yang, Shengda Fan +7
Experience internalization converts contextual experience from past interactions into reusable parametric capability, offering a promising path toward continual learning in large l…
SIRI: Self-Internalizing Reinforcement Learning with Intrinsic Skills for LLM Agent Training
Zhongyu He, Yuanfan Li, Fei Huang +9
Long-horizon LLM agents can benefit from reusable skills, yet existing skill-based methods often rely on external skill generators during training or persistent skill retrieval at…
Skill or Skip? Learning Selective Skill Invocation in Agentic Tasks via Dual-Granularity Preference Learning
Chishui Chen, Jiaye Lin, Te Sun +6
Agent skills are callable procedural modules that provide reusable knowledge and execution policies for complex agentic tasks. However, existing methods mainly focus on selecting r…
From to : Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight
Xiaoliang Fu, Jiaye Lin, Yangyi Fang +6
Reinforcement Learning with Verifiable Rewards (RLVR) has catalyzed a leap in Large Language Model (LLM) reasoning, yet its optimization dynamics remain fragile. Standard algorithm…
How to Allocate, How to Learn? Dynamic Rollout Allocation and Advantage Modulation for Policy Optimization
Yangyi Fang, Jiaye Lin, Xiaoliang Fu +6
Reinforcement Learning with Verifiable Rewards (RLVR) has proven effective for Large Language Model (LLM) reasoning, yet current methods face key challenges in resource allocation…
MASPO: Unifying Gradient Utilization, Probability Mass, and Signal Reliability for Robust and Sample-Efficient LLM Reasoning
Xiaoliang Fu, Jiaye Lin, Yangyi Fang +7
Existing Reinforcement Learning with Verifiable Rewards (RLVR) algorithms, such as GRPO, rely on rigid, uniform, and symmetric trust region mechanisms that are fundamentally misali…