Showing cs.AIShow all
3 papers · 1 filter
cs.AI2026
SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents
Pujun Zheng, Zixin Shang, Shufan Jiang +5
SWE-Bench Pro has emerged as a standard benchmark for evaluating software engineering agents on challenging repository-level tasks. However, our analysis work show that its evaluat…
cs.AI2026
AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
Kai Chen, Zichen Ding, Jiaye Ge +20
As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical. However, current evaluation pipelines remain highly…
cs.AI2026
When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents
Dongsheng Zhu, Xuchen Ma, Yucheng Shen +5
Existing benchmarks evaluate Tool-Integrated Reasoning (TIR) in LLMs on idealized ''happy paths'', largely overlooking real-world tool failures. We introduce ToolMaze, a benchmark…