5 papers · 1 filter
No More Stale Feedback: Co-Evolving Critics for Open-World Agent Learning
Zhicong Li, Lingjie Jiang, Yulan Hu +7
Critique-guided reinforcement learning (RL) has emerged as a powerful paradigm for training LLM agents by augmenting sparse outcome rewards with natural-language feedback. However,…
SPPO: Sequence-Level PPO for Long-Horizon Reasoning Tasks
Tianyi Wang, Yixia Li, Long Li +6
Proximal Policy Optimization (PPO) is central to aligning Large Language Models (LLMs) in reasoning tasks with verifiable rewards. However, standard token-level PPO struggles in th…
From Abstract to Contextual: What LLMs Still Cannot Do in Mathematics
Bowen Cao, Dongdong Zhang, Yixia Li +8
Large language models now solve many benchmark math problems at near-expert levels, yet this progress has not fully translated into reliable performance in real-world applications.…
Rethinking the Role of Entropy in Optimizing Tool-Use Behaviors for Large Language Model Agents
Zeping Li, Hongru Wang, Yiwen Zhao +7
Tool-using agents based on Large Language Models (LLMs) excel in tasks such as mathematical reasoning and multi-hop question answering. However, in long trajectories, agents often…
Anchored Policy Optimization: Mitigating Exploration Collapse Via Support-Constrained Rectification
Tianyi Wang, Long Li, Hongcan Guo +5
Reinforcement Learning with Verifiable Rewards (RLVR) is increasingly viewed as a tree pruning mechanism. However, we identify a systemic pathology termed Recursive Space Contracti…