9 papers
How Much, Then Where: Credit-Conserving Action-to-Token Allocation for Multi-Turn Agent Reinforcement Learning
Lichao Ma, Yang Sun, Shuaitao Zhao +9
Credit assignment in multi-turn agent reinforcement learning operates at two levels: assigning trajectory-level credit to actions and distributing each action's credit across its t…
ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation
Zhenghan Chen, Zekai Shao, Lidan Tan +10
Annotations are essential to communicative visualization, helping explain data, emphasize key findings, and guide attention. While multimodal large language models (MLLMs) offer ne…
Group-Reflective Self-Distillation for Agentic Reinforcement Learning
Binbin Zheng, Zijun Xie, Guanqun Zhao +4
Reinforcement learning with verifiable rewards (RLVR) is effective for training large language model agents. However, terminal rewards provide only coarse trajectory-level supervis…
SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting
Binbin Zheng, Xing Ma, Yiheng Liang +6
On-policy reinforcement learning has become the dominant paradigm for reasoning alignment in large language models, yet its sparse, outcome-level rewards make token-level credit as…
Placing Puzzle Pieces Where They Matter: A Question Augmentation Framework for Reinforcement Learning
Yangyi Fang, Jiaye Lin, Xiaoliang Fu +2
Reinforcement learning has become a powerful approach for enhancing large language model reasoning, but faces a fundamental dilemma: training on easy problems can cause overfitting…
From to : Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight
Xiaoliang Fu, Jiaye Lin, Yangyi Fang +6
Reinforcement Learning with Verifiable Rewards (RLVR) has catalyzed a leap in Large Language Model (LLM) reasoning, yet its optimization dynamics remain fragile. Standard algorithm…
Proximity-Based Multi-Turn Optimization: Practical Credit Assignment for LLM Agent Training
Yangyi Fang, Jiaye Lin, Xiaoliang Fu +4
Multi-turn LLM agents are becoming pivotal to production systems, spanning customer service automation, e-commerce assistance, and interactive task management, where accurately dis…
How to Allocate, How to Learn? Dynamic Rollout Allocation and Advantage Modulation for Policy Optimization
Yangyi Fang, Jiaye Lin, Xiaoliang Fu +6
Reinforcement Learning with Verifiable Rewards (RLVR) has proven effective for Large Language Model (LLM) reasoning, yet current methods face key challenges in resource allocation…
MASPO: Unifying Gradient Utilization, Probability Mass, and Signal Reliability for Robust and Sample-Efficient LLM Reasoning
Xiaoliang Fu, Jiaye Lin, Yangyi Fang +7
Existing Reinforcement Learning with Verifiable Rewards (RLVR) algorithms, such as GRPO, rely on rigid, uniform, and symmetric trust region mechanisms that are fundamentally misali…