6 papers
Stale Instruction Files Override the Code in Coding Agents: A One-Sentence Fix
Xiaonan Xu, Wenjing Wu
Purpose: Coding agents follow a project's instruction file, such as CLAUDE.md or AGENTS.md, even when it disagrees with the existing code and even when they notice the disagreement…
What Aggregate Scores Miss: Measuring Item-Level Regressions in Commercial LLM API Migrations
Xiaonan Xu, Wenjing Wu
Context: Software systems that depend on commercial large language model APIs must migrate to successor versions when vendors deprecate older models. Migration decisions typically…
Low Reasoning Effort Is Enough for Routine Office Work by Language-Model Agents
Xiaonan Xu, Wenjing Wu
Purpose: Developers choose how much a language-model agent reasons before it acts. Some pick a high level for fear that a low one is not enough, and pay for it in tokens, time and…
Validation Evidence in LLM Repair Agents: How Much of What Passes Actually Tests the Bug?
Xiaonan Xu, Wenjing Wu
When a repair agent runs a test and sees it pass, the result is treated as evidence about the reported defect. We measure how often that treatment is warranted. BSG-VA (buggy-state…
Shortening agent skills leaves LLM agent cost unchanged
Xiaonan Xu, Wenjing Wu
Agent skills are often shortened, by compression, rewriting with a language model or partial loading, to cut the cost of running LLM agents. We tested whether this works on 40 Skil…
Skill Availability and Presentation Granularity in Large-Language-Model Agents: A Controlled SkillsBench Study
Xiaonan Xu, Wenjing Wu
Skill documents provide procedural knowledge to large-language-model agents at inference time. This article studies whether the presentation granularity of controlled skill knowled…