3 citations · 4 across the 7 of their papers we have counts for
8 papers
Diagnosing with Insights: Structured Analysis of Agent Failures via Behavioral Abstractions
Jiayi Bi, Yanjie Gao, Yuanmin Xie +4
With the proliferation of LLM agents, the ability to understand and diagnose failures in agents is essential to achieving superior effectiveness and trustworthiness. As agent failu…
Agent Skills Can Be Harmful: An Empirical Study of Skill-Induced Failures in LLM Agents
Gen Dong, Yanjie Gao, Liqun Li +3
Agent skills are the de facto mechanism for extending LLM agents with reusable guidance. A skill can shape the agent's task execution, including planning, tool use, problem-solving…
Towards a Systems Foundation for Agentic Cloud Management
Minghao Li, Ziqian Liu, Ziyu Mao +5
Agentic cloud management is emerging as a practice to automate laborious operations, minimize toil, and improve responsiveness. Despite the rapid development of autonomous manageme…
SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure Scenarios
Jackson Clark, Yiming Su, Saad Mohammad Rafid Pial +5
AI agents are increasingly used to diagnose and mitigate failures in production systems, known as agentic Site Reliability Engineering (SRE). Current SRE benchmarks are limited to…
STRATUS: A Multi-agent System for Autonomous Reliability Engineering of Modern Clouds
Yinfang Chen, Jiaqi Pan, Jackson Clark +7
In cloud-scale systems, failures are the norm. A distributed computing cluster exhibits hundreds of machine failures and thousands of disk failures; software bugs and misconfigurat…
An Empirical Study of Production Incidents in Generative AI Cloud Services
Haoran Yan, Yinfang Chen, Minghua Ma +10
The ever-increasing demand for generative artificial intelligence (GenAI) has motivated cloud-based GenAI services such as Azure OpenAI Service and Amazon Bedrock. Like any large-s…
ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Tasks
Saurabh Jha, Rohan Arora, Yuji Watanabe +40
Realizing the vision of using AI agents to automate critical IT tasks depends on the ability to measure and understand effectiveness of proposed solutions. We introduce ITBench, a…
Configuration Validation with Large Language Models
Xinyu Lian, Yinfang Chen, Runxiang Cheng +4
Misconfigurations are major causes of software failures. Existing practices rely on developer-written rules or test cases to validate configurations, which are expensive. Machine l…