13 papers
CreepScope: Measuring Post-Use Authorization Exposure in Tool-Using Agents
Haoran Yu, Lifei Liu, Danping Zhang +4
Authorization checks at individual tool calls do not reveal how long earlier authority remains usable. We study this lifecycle failure as privilege creep: a capability stays reacha…
Noise Floor Audit for Agent Benchmarks
Yihang Chen, Pin Qian, Su Wang +4
We audit measurement variability for 3 native tool-calling endpoints across 2 providers on the official BFCL multiple and parallel categories, using matched AST grading. At tempera…
Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classification
Yihang Chen, Pin Qian, Su Wang +4
Personalized agents must decide whether retrieved user memory should be used, ignored, updated, or queried before it affects a current task. We use this setting to develop an empir…
BAP-SQL: Budget-Aware Observation Planning for Agentic Text-to-SQL
Chong Peng, Pin Qian, Su Wang +2
Tool-using agents do not merely consume observations: their actions determine what arrives next. In agentic text-to-SQL, a broad query can spend context and database work before us…
When Should Active RAG Retrieve? A Budget-Aware Evaluation of Utility, Calibration, and Cost
Pin Qian, Su Wang, Chong Peng +5
Active RAG systems decide when to retrieve external knowledge during generation, making them a budget-sensitive case of agentic RAG and self-adaptive retrieval. Yet evaluations oft…
Toward User-Conditioned Evaluation of Personal LLM Agents under Temporal Interventions
Pin Qian, Su Wang, Yihang Chen +5
Personal agents maintain memories, learned skills, tool configurations, and policy state that evolve with each user. Existing agent benchmarks often evaluate these capabilities in…
Phantom Guardrails: When Self-Improving Agent Harnesses Fix Failures That Never Happened
Su Wang, Pin Qian, Yifan Lin +5
Self-improving AI agents are designed to learn from their mistakes. We show they can also hallucinate mistakes that never happened. We study this failure mode in automated harness…
Operational Reframing and Approval-Framed Delegation in Multi-Agent LLM Safety
Lifei Liu, Haoran Yu, Xiaochong Jiang +3
Safety evaluations of multi-agent LLM systems often compare a direct prompt with a planner-executor pipeline and report the difference as a single "pipeline effect." We argue that…
Habituation at the Gate: Rising Approval and Declining Scrutiny in Human Review of AI Agent Code
Haoran Yu, Lifei Liu, Xiaochong Jiang +4
As AI coding agents (e.g., GitHub Copilot, Devin, OpenAI Codex, Cursor) submit pull requests to open-source repositories at scale, a key question arises: do human reviewers gradual…
Beyond Simpson's Paradox: A Cascade of Confounders in AI Agent Pull-Request Co-Authorship
Haoran Yu, Xiaochong Jiang, Lifei Liu +3
Pooled across five AI coding agents, pull requests (PRs) with a human Co-Authored-By trailer merge less often than purely-autonomous ones (53.8% vs. 79.8%) -- yet this aggregate fi…
When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems
Su Wang, Pin Qian, Yihang Chen +6
LLM agents increasingly rely on community-contributed skills that expand an agent's operational capability set. We study a core safety problem in agentic AI systems: whether indivi…
Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG
Pin Qian, Su Wang, Xiaoyuan Wang +7
Cited RAG evaluation often treats visible sources as a grounding signal, but a real, topically relevant citation can still under-warrant the attached wording. We study this diagnos…
Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict
Yihang Chen, Pin Qian, Su Wang +4
Retrieval-Augmented Generation (RAG) is usually evaluated by whether the final answer is correct. Under knowledge conflict, this hides a key question: did the model follow retrieve…