From the 1 of 5 linked papers with an AI index.
5 papers
Specula: Scaling formal specifications for autonomous model checking of system code
Qian Cheng, Saad Mohammad Rafid Pial, Ruize Tang +6
Specula is an autonomous system that uses large language model agents to generate TLA+ specifications for complex system code and then applies model checking to discover bugs.
SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure Scenarios
Jackson Clark, Yiming Su, Saad Mohammad Rafid Pial +5
AI agents are increasingly used to diagnose and mitigate failures in production systems, known as agentic Site Reliability Engineering (SRE). Current SRE benchmarks are limited to…
STRATUS: A Multi-agent System for Autonomous Reliability Engineering of Modern Clouds
Yinfang Chen, Jiaqi Pan, Jackson Clark +7
In cloud-scale systems, failures are the norm. A distributed computing cluster exhibits hundreds of machine failures and thousands of disk failures; software bugs and misconfigurat…
SysMoBench: Evaluating AI on Formally Modeling Complex Real-World Systems
Qian Cheng, Ruize Tang, Emilie Ma +7
Formal models are essential to specifying large, complex computer systems and verifying their correctness, but are notoriously expensive to write and maintain. Recent advances in g…
Neuro-Symbolic Verification on Instruction Following of LLMs
Yiming Su, Kunzhao Xu, Yanjie Gao +4
A fundamental problem of applying Large Language Models (LLMs) to important applications is that LLMs do not always follow instructions, and violations are often hard to observe or…