9 papers
Lightweight, Rubric-Guided Trajectory Evaluation for Production AI Agents
Linh-An Phan, MingXue Wang, Guangyu Wu +3
Trajectory evaluation is essential for improving the reliability of LLM-based agents, but production use makes it expensive to run repeatedly. Modern agents generate long traces co…
The Nuts and Bolts of Natural Language to SQL Translation: A Systematic Analysis of Model Pipeline Optimisation Approaches and their Interactions
Filip Klubicka, Vasudevan Nedumpozhimana, Sneha Rautmare +3
In the age of large language models, Natural Language to SQL (NL2SQL) translation remains an open problem with many useful applications. We explore interactions between several NL2…
Log-Insight: Automating Microservice Incident Diagnosis via Neuro-Symbolic Log Analysis
Carlos Garcia-Hernandez, Aymane Abdali, Guangyu Wu +4
Diagnosing production incidents in large-scale microservice systems is time-critical for Site Reliability Engineers (SREs). A single 30-minute incident window in our deployment can…
Efficient Multi-Cohort Inference for Long-Term Effects and Lifetime Value in A/B Testing with User Learning
Dario Simionato, Andrea Tonon, Mingxue Wang +3
In streaming platforms churn is extremely costly, yet A/B tests are typically evaluated using outcomes observed within a limited experimental horizon. Even when both short- and pre…
Large Causal Models for Temporal Causal Discovery
Nikolaos Kougioulis, Nikolaos Gkorgkolis, MingXue Wang +4
Causal discovery for both cross-sectional and temporal data has traditionally followed a dataset-specific paradigm, where a new model is fitted for each individual dataset. Such an…
Adversarial Causal Tuning for Realistic Time-series Generation
Nikolaos Gkorgkolis, Nikolaos Kougioulis, MingXue Wang +4
We address the problem of generating simulated, yet realistic, time-series data from a causal model with the same observational and interventional distributions as a given real dat…
RADICE: Causal Graph Based Root Cause Analysis for System Performance Diagnostic
Andrea Tonon, Meng Zhang, Bora Caglayan +4
Root cause analysis is one of the most crucial operations in software reliability regarding system performance diagnostic. It aims to identify the root causes of system performance…
BIS: NL2SQL Service Evaluation Benchmark for Business Intelligence Scenarios
Bora Caglayan, Mingxue Wang, John D. Kelleher +4
NL2SQL (Natural Language to Structured Query Language) transformation has seen wide adoption in Business Intelligence (BI) applications in recent years. However, existing NL2SQL be…
SHREC: a SRE Behaviour Knowledge Graph Model for Shell Command Recommendations
Andrea Tonon, Bora Caglayan, MingXue Wang +3
In IT system operations, shell commands are common command line tools used by site reliability engineers (SREs) for daily tasks, such as system configuration, package deployment, a…