8 papers
Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories
Jiaming Wang, Ziteng Feng, Jiangtao Wu +8
Deep-research agents solve tasks through long trajectories of search, tool use, evidence inspection, and answer synthesis. Evaluation based on final answers shows whether an agent…
CodeTracer: Towards Traceable Agent States
Han Li, Yifan Yao, Letian Zhu +13
Code agents are advancing rapidly, but debugging them is becoming increasingly difficult. As frameworks orchestrate parallel tool calls and multi-stage workflows over complex tasks…
Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence?
Qianshan Wei, Yishan Yang, Siyi Wang +12
Multimodal Large Language Models (MLLMs) are evolving from passive observers into active agents, solving problems through Visual Expansion (invoking visual tools) and Knowledge Exp…
ContextBench: A Benchmark for Context Retrieval in Coding Agents
Han Li, Letian Zhu, Bohan Zhang +7
LLM-based coding agents have shown strong performance on automated issue resolution benchmarks, yet existing evaluations largely focus on final task success, providing limited insi…
SOP-Maze: Evaluating Large Language Models on Complicated Business Standard Operating Procedures
Jiaming Wang, Zhe Tang, Zehao Jin +5
As large language models (LLMs) are widely deployed as domain-specific agents, many benchmarks have been proposed to evaluate their ability to follow instructions and make decision…
Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs' Instruction Following Capability
Jiaming wang, Yunke Zhao, Peng Ding +8
The capability to precisely adhere to instructions is a cornerstone for Large Language Models (LLMs) to function as dependable agents in real-world scenarios. However, confronted w…