4 papers
A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis
Yuanhong Cai, Xiaohui Nie, Kanglin Yin +8
LLM-based agents are reshaping microservice operations into AgentOps, where benchmarks are key to evaluating failure diagnosis over multimodal observability data. However, existing…
UModel: An Agent-Ready Observability Data Modeling Method at Scale
Changhua Pei, Zheyuan Li, Zexin Wang +10
When networked system failures occur, automatically performing Root Cause Analysis (RCA) using observability data is critical for ensuring networked system reliability. Recently, L…
TrioXpert: An Automated Incident Management Framework for Microservice System
Yongqian Sun, Yu Luo, Xidao Wen +5
Automated incident management plays a pivotal role in large-scale microservice systems. However, many existing methods rely solely on single-modal data (e.g., metrics, logs, and tr…
OpsEval: A Comprehensive IT Operations Benchmark Suite for Large Language Models
Yuhe Liu, Changhua Pei, Longlong Xu +13
Information Technology (IT) Operations (Ops), particularly Artificial Intelligence for IT Operations (AIOps), is the guarantee for maintaining the orderly and stable operation of e…