1 citations · 1 across the 15 of their papers we have counts for
16 papers
ICAE-Bench: Evaluating Coding Agents as Interactive Project Builders
Zhongyuan Peng, Dan Huang, Chuyu Zhang +8
The recent emergence of vibe-coding workflows is changing what coding agents are expected to do. Instead of merely completing code under fully specified instructions, agents are in…
DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Tasks
Jingxuan Han, Wei Liu, Mingyang Zhu +8
Search Agents (SAs) typically leverage large language models (LLMs) to support complex information-seeking tasks by autonomously exploring web sources and synthesizing information…
Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinement
Xin Wang, Liangtai Sun, Yaoming Zhu +8
Existing code-generation benchmarks score a single mapping from a complete prompt to a one-shot output. However, real web development is different. Users seldom write a full spec a…
SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystems
Linyue Pan, Yaoming Zhu, Lin Qiu +2
Self-improving language agents are typically evaluated in isolation: an agent attempts a task, receives feedback, and iteratively refines its own behavior. Yet agents increasingly…
AgentEscapeBench: Evaluating Out-of-Domain Tool-Grounded Reasoning in LLM Agents
Zhengkang Guo, Yiyang Li, Lin Qiu +7
As LLM-based agents increasingly rely on external tools, it is important to evaluate their ability to sustain tool-grounded reasoning beyond familiar workflows and short-range inte…
SWE-Cycle: Benchmarking Code Agents across the Complete Issue Resolution Cycle
Hao Guan, Lingyue Fu, Shao Zhang +8
As autonomous code agents move toward end-to-end software development, evaluating their practical autonomy becomes critical. Current benchmarks hide friction by testing agents in p…
LongCat-Next: Lexicalizing Modalities as Discrete Tokens
Meituan LongCat Team, Bin Xiao, Chao Wang +86
The prevailing Next-Token Prediction (NTP) paradigm has driven the success of large language models through discrete autoregressive modeling. However, contemporary multimodal syste…
AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations
Cheng Jiayang, Dongyu Ru, Lin Qiu +4
Long-horizon interactions between users and LLM-based assistants necessitate effective memory management, yet current approaches face challenges in training and evaluation of memor…
CATArena: Evaluating Evolutionary Capabilities of Code Agents via Iterative Tournaments
Lingyue Fu, Xin Ding, Linyue Pan +9
Current evaluation for Large Language Model (LLM) code agents predominantly focus on generating functional code in single-turn scenarios, which fails to evaluate the agent's capabi…
Automatically Benchmarking LLM Code Agents through Agent-Driven Annotation and Evaluation
Lingyue Fu, Bolun Zhang, Hao Guan +7
Recent advances in code agents have enabled automated software development at the project level, supported by large language models (LLMs). However, existing benchmarks for code ag…
Instance-level Randomization: Toward More Stable LLM Evaluations
Yiyang Li, Yonghuang Wu, Ying Luo +5
Evaluations of large language models (LLMs) suffer from instability, where small changes of random factors such as few-shot examples can lead to drastic fluctuations of scores and…
CoreCodeBench: Decoupling Code Intelligence via Fine-Grained Repository-Level Tasks
Lingyue Fu, Hao Guan, Bolun Zhang +10
The evaluation of Large Language Models (LLMs) for software engineering has shifted towards complex, repository-level tasks. However, existing benchmarks predominantly rely on coar…
OIBench: Benchmarking Strong Reasoning Models with Olympiad in Informatics
Yaoming Zhu, Junxin Wang, Yiyang Li +8
As models become increasingly sophisticated, conventional algorithm benchmarks are increasingly saturated, underscoring the need for more challenging benchmarks to guide future imp…
Audio Turing Test: Benchmarking the Human-likeness of Large Language Model-based Text-to-Speech Systems in Chinese
Xihuai Wang, Ziyi Zhao, Siyu Ren +9
Recent advances in large language models (LLMs) have significantly improved text-to-speech (TTS) systems, enhancing control over speech style, naturalness, and emotional expression…
Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration
Shao Zhang, Xihuai Wang, Wenhao Zhang +10
Agents built on large language models (LLMs) have excelled in turn-by-turn human-AI collaboration but struggle with simultaneous tasks requiring real-time interaction. Latency issu…
Understanding and Optimizing Agentic Workflows via Shapley value
Yingxuan Yang, Bo Huang, Siyuan Qi +14
Agentic workflows have become the dominant paradigm for building complex AI systems, orchestrating specialized components, such as planning, reasoning, action execution, and reflec…