3 citations · 6 across the 14 of their papers we have counts for
6 papers · 1 filter
Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI
Jiaqi Shao, Hanck Chen, Wei Zhang +2
Agent benchmarks increasingly evaluate repository editing, web research, terminal use, and long-horizon interaction. Their scores support capability claims only when the evaluation…
Do More Agents Help? Evaluating Accuracy and Cost in LLM Multi-Agent Workflows
Yuhang Fu, Ruishan Fang, Jiaqi Shao +5
Choosing whether to replace a single-agent controller requires evidence on both task performance and inference expenditure. We present BenchAgent, a shared evaluation framework, an…
When Stored Evidence Stops Being Usable: Scale-Conditioned Evaluation of Agent Memory
Jiaqi Shao, Yiyi Lu, Yunzhen Zhang +1
Memory-agent evaluations report fixed-snapshot accuracy or retrieval quality, but these scores do not show whether evidence remains usable as irrelevant sessions (sessions not anno…
Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agents
Jiaqi Shao, Yuxiang Lin, Munish Prasad Lohani +2
Recent work has explored training Large Language Model (LLM) search agents with reinforcement learning (RL) for open-domain question answering (QA). However, most evaluations focus…
MorphAgent: Empowering Agents through Self-Evolving Profiles and Decentralized Collaboration
Siyuan Lu, Jiaqi Shao, Bing Luo +1
Large Language Model (LLM) based multi-agent systems (MAS) have shown promise in tackling complex tasks, but often rely on predefined roles and centralized coordination, limiting t…
Federated Unlearning: a Perspective of Stability and Fairness
Jiaqi Shao, Tao Lin, Xuanyu Cao +1
This paper explores the multifaceted consequences of federated unlearning (FU) with data heterogeneity. We introduce key metrics for FU assessment, concentrating on verification, g…