7 papers
What Aggregate Scores Miss: Measuring Item-Level Regressions in Commercial LLM API Migrations
Xiaonan Xu, Wenjing Wu
Context: Software systems that depend on commercial large language model APIs must migrate to successor versions when vendors deprecate older models. Migration decisions typically…
Low Reasoning Effort Is Enough for Routine Office Work by Language-Model Agents
Xiaonan Xu, Wenjing Wu
Purpose: Developers choose how much a language-model agent reasons before it acts. Some pick a high level for fear that a low one is not enough, and pay for it in tokens, time and…
Validation Evidence in LLM Repair Agents: How Much of What Passes Actually Tests the Bug?
Xiaonan Xu, Wenjing Wu
When a repair agent runs a test and sees it pass, the result is treated as evidence about the reported defect. We measure how often that treatment is warranted. BSG-VA (buggy-state…
Shortening agent skills leaves LLM agent cost unchanged
Xiaonan Xu, Wenjing Wu
Agent skills are often shortened, by compression, rewriting with a language model or partial loading, to cut the cost of running LLM agents. We tested whether this works on 40 Skil…
Skill Availability and Presentation Granularity in Large-Language-Model Agents: A Controlled SkillsBench Study
Xiaonan Xu, Wenjing Wu
Skill documents provide procedural knowledge to large-language-model agents at inference time. This article studies whether the presentation granularity of controlled skill knowled…
Large Language Model-Driven Cross-Domain Orchestration Using Multi-Agent Workflow
Xiaonan Xu, Haoshuo Chen, Jesse E. Simsarian +5
We showcase an application that leverages multiple agents, powered by large language models and integrated tools, to collaboratively solve complex network operation tasks across va…
Digital Twin of a Network and Operating Environment Using Augmented Reality
Haoshuo Chen, Xiaonan Xu, Jesse E. Simsarian +9
We demonstrate the digital twin of a network, network elements, and operating environment using machine learning. We achieve network card failure localization and remote collaborat…