280 citations · 1.4k across the 75 of their papers we have counts for
5 papers · 1 filter
VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent Guardrails
Duy Nguyen, Lesly Miculicich, Bhavana Dalvi Mishra +5
LLM agents increasingly operate over sensitive resources, such as private documents and mutable system states, which can create serious safety and security risks. Runtime guardrail…
SpecGuard: Proving a Task Is Broken Before the Agent Cheats
Param Biyani, Krishnamurthy Dvijotham
As autonomous coding agents get increasingly deployed, the risk that accidental or adversarially injected misspecifications in tasks lead to dangerous agent behavior is critical to…
Efficient Symbolic Reasoning for Neural-Network Verification
Zi Wang, Somesh Jha, Krishnamurthy +1
The neural network has become an integral part of modern software systems. However, they still suffer from various problems, in particular, vulnerability to adversarial attacks. In…
Role of Human-AI Interaction in Selective Prediction
Elizabeth Bondi, Raphael Koster, Hannah Sheahan +5
Recent work has shown the potential benefit of selective prediction systems that can learn to defer to a human when the predictions of the AI are unreliable, particularly to improv…
Safe Exploration in Continuous Action Spaces
Gal Dalal, Krishnamurthy Dvijotham, Matej Vecerik +3
We address the problem of deploying a reinforcement learning (RL) agent on a physical system such as a datacenter cooling unit or robot, where critical constraints must never be vi…