10 papers
ArenaFlow: From Trajectory Ranking to Hierarchical Credit Propagation for Open-Ended Agent RL
Qiang Zhang, Ruixue Ding, Fanrui Zhang +9
Reinforcement learning has substantially improved large language model (LLM) agents in verifiable domains, but remains difficult to apply to open-ended agent tasks, where solutions…
CHART: A Harness-Rotation Curriculum for Harness-Robust Search Agents
Xinlu Zhang, Ying-Chun Lin, Zhihan Zhang +2
Search agents are usually trained under a single harness. But once an agent is deployed in a real application, its harness is frequently updated (e.g., a rewritten system prompt) t…
ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning
Fanrui Zhang, Ruixue Ding, Qiang Zhang +13
Training open-ended agents via reinforcement learning (RL) is hindered by the lack of verifiable gold answers and scalable rubrics. Moreover, even near the model's capability bound…
Two-Fidelity Best-Action Identification for Stochastic Minimax Tree
Peter Chen, Xi Chen
We study fixed-confidence best-action identification (BAI) in stochastic minimax trees. This problem is increasingly relevant in modern AI planning, where deep minimax search and M…
What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search
Xinhao Zhang, Xi Chen, François Portet +1
Recent work has demonstrated the promise of orchestrating large language models (LLMs) within evolutionary and agentic optimization systems. However, the mechanisms driving these o…
AgentCollab: A Self-Evaluation-Driven Collaboration Paradigm for Efficient LLM Agents
Wenbo Gao, Renxi Liu, Xian Wang +8
Autonomous agents powered by large language models (LLMs) perform complex tasks through long-horizon reasoning and tool interaction, where a fundamental trade-off arises between ex…
Reward-free Alignment for Conflicting Objectives
Peter Chen, Xiaopeng Li, Xi Chen +1
Direct alignment methods are increasingly used to align large language models (LLMs) with human preferences. However, many real-world alignment problems involve multiple conflictin…
Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward
Peter Chen, Xiaopeng Li, Ziniu Li +3
This paper examines the exploration-exploitation trade-off in reinforcement learning with verifiable rewards (RLVR), a framework for improving the reasoning of Large Language Model…
Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO
Peter Chen, Xiaopeng Li, Ziniu Li +2
Reinforcement learning (RL) has proven effective in strengthening the reasoning capabilities of large language models (LLMs). A widely adopted method, Group Relative Policy Optimiz…
ComPO: Preference Alignment via Comparison Oracles
Peter Chen, Xi Chen, Wotao Yin +1
Direct alignment methods are increasingly used for aligning large language models (LLMs) with human preferences. However, these methods suffer from the issues of verbosity and like…