8 papers
PEARL: Process-Experience Aware Reinforcement Learning for Multi-Turn Agents
Jie Yang, Mianqiu Huang, Yiwei Zhao +3
Terminal rewards make multi-turn agents trainable without step-level annotations, but group-relative advantages provide little guidance about which intermediate decisions to adjust…
VehicleArena: A Realistic Urban Environment for Multi-Agent Driving
Jie Yang, Jiajun Chen, Jiazheng Zhou +4
Real-world embodied agents often pursue independent objectives within a shared physical environment, where their actions can alter the conditions faced by others. Existing benchmar…
TraceDance: An Automated System for Building Agent Behavior Benchmarks from Real-World Agent Deployment Traces
Dehai Min, Daoan Zhang, Yiming Zeng +13
An agent can complete a task while exhibiting undesirable behavior during execution. Developers need tests for the specific behaviors encountered in deployment, beyond fixed benchm…
ContextWeave: A Real-World Workflow Benchmark
Bo Wang, Yuqian Yao, Enxi Wang +25
Memory is essential as language agents move from isolated tasks to long-horizon, stateful workflows, yet existing evaluations often reduce it to retrieval or question answering. We…
Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews
Bowen Li, Haochen Ma, Yuxin Wang +5
The rapid adoption of Large Language Models (LLMs) has spurred interest in automated peer review; however, progress is currently stifled by benchmarks that treat reviewing primaril…
ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development
Jie Yang, Honglin Guo, Li Ji +11
The evolution of Large Language Models (LLMs) into autonomous agents has expanded the scope of AI coding from localized code generation to complex, repository-level, and execution-…
VehicleWorld: A Highly Integrated Multi-Device Environment for Intelligent Vehicle Interaction
Jie Yang, Jiajun Chen, Zhangyue Yin +7
Intelligent vehicle cockpits present unique challenges for API Agents, requiring coordination across tightly-coupled subsystems that exceed typical task environments' complexity. T…
FamilyTool: A Multi-hop Personalized Tool Use Benchmark
Yuxin Wang, Yiran Guo, Yining Zheng +7
The integration of tool learning with Large Language Models (LLMs) has expanded their capabilities in handling complex tasks by leveraging external tools. However, existing benchma…