8 papers · 1 filter
ICAE-Bench: Evaluating Coding Agents as Interactive Project Builders
Zhongyuan Peng, Dan Huang, Chuyu Zhang +8
The recent emergence of vibe-coding workflows is changing what coding agents are expected to do. Instead of merely completing code under fully specified instructions, agents are in…
DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Tasks
Jingxuan Han, Wei Liu, Mingyang Zhu +8
Search Agents (SAs) typically leverage large language models (LLMs) to support complex information-seeking tasks by autonomously exploring web sources and synthesizing information…
SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystems
Linyue Pan, Yaoming Zhu, Lin Qiu +2
Self-improving language agents are typically evaluated in isolation: an agent attempts a task, receives feedback, and iteratively refines its own behavior. Yet agents increasingly…
AgentEscapeBench: Evaluating Out-of-Domain Tool-Grounded Reasoning in LLM Agents
Zhengkang Guo, Yiyang Li, Lin Qiu +7
As LLM-based agents increasingly rely on external tools, it is important to evaluate their ability to sustain tool-grounded reasoning beyond familiar workflows and short-range inte…
CATArena: Evaluating Evolutionary Capabilities of Code Agents via Iterative Tournaments
Lingyue Fu, Xin Ding, Linyue Pan +9
Current evaluation for Large Language Model (LLM) code agents predominantly focus on generating functional code in single-turn scenarios, which fails to evaluate the agent's capabi…
Understanding and Optimizing Agentic Workflows via Shapley value
Yingxuan Yang, Bo Huang, Siyuan Qi +14
Agentic workflows have become the dominant paradigm for building complex AI systems, orchestrating specialized components, such as planning, reasoning, action execution, and reflec…