6 papers · 1 filter
EvoCode-Bench: Evaluating Coding Agents in Multi-Turn Iterative Interactions
Haiyang Shen, Xuanzhong Chen, Wendong Xu +3
Coding agents are increasingly used as iterative development partners, but most benchmarks still evaluate one specification followed by one final assessment. This leaves out a basi…
SGR-Bench: Benchmarking Search Agents on State-Gated Retrieval
Ningyuan Li, Haiyang Shen, Mugeng Liu +4
Recent advances in large language models and tool-using agents have expanded the range of benchmarked web tasks. Yet an important class of specialized retrieval tasks remains under…
Teaching AI Through Benchmark Construction: QuestBench as a Course-Based Practice for Accountable Knowledge Work
Haiyang Shen, Jiuzheng Wang, Taian Guo +9
As AI becomes part of everyday learning, many courses teach students to use it mainly as a productivity tool: how to prompt, search, summarize, write, code, and use tools more effi…
MindLoom: Composing Thought Modes for Frontier-Level Reasoning Data Synthesis
Haiyang Shen, Taian Guo, Xuanzhong Chen +11
Although LLMs have made substantial progress in reasoning, systematically producing frontier-level reasoning data remains difficult. Existing synthesis methods often have limited v…
DeepWeb-Bench: A Deep Research Benchmark Demanding Massive Cross-Source Evidence and Long-Horizon Derivation
Sixiong Xie, Zhuofan Shi, Haiyang Shen +8
Deep research, in which an agent searches the open web, collects evidence, and derives an answer through extended reasoning, is a prominent use case for frontier language models. F…
M3-BENCH: Process-Aware Evaluation of LLM Agents' Social Behaviors in Mixed-Motive Games
Sixiong Xie, Zhuofan Shi, Haiyang Shen +2
Existing benchmarks for LLM agents' social behavior typically focus on a single capability dimension and evaluate only behavioral outcomes, overlooking process signals from reasoni…