5 papers · 1 filter
Self-Healing Harness for Runtime Oversight of Agent Self-Modification
Sina Tayebati, Divake Kumar, Nastaran Darabi +2
LLM agents can change their own future behavior, raising a basic control question of which self-generated changes should be allowed to persist. We formulate this as admission contr…
CURA: Certified Runtime Alarms for Computer-Use Agents
Divake Kumar, Sina Tayebati, Devashri Naik +5
Self-report is the cheapest oversight channel a deployer has, and on capable computer-use agents (CUAs) it fails precisely where oversight matters. On 361 OSWorld tasks our pipelin…
Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets
Divake Kumar, Sina Tayebati, Devashri Naik +5
Computer-use agents turn vision-language model (VLM) predictions into executable GUI clicks, so reliable uncertainty estimates are essential for rejection, calibration, miss-severi…
VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation
Divake Kumar, Sina Tayebati, Devashri Naik +2
Vision-language models (VLMs) are increasingly used as automated judges for multimodal systems, yet their scores provide no indication of reliability. We study this problem through…
TRACER: Trajectory Risk Aggregation for Critical Episodes in Agentic Reasoning
Sina Tayebati, Divake Kumar, Nastaran Darabi +3
Estimating uncertainty for AI agents in real-world multi-turn tool-using interaction with humans is difficult because failures are often triggered by sparse critical episodes (e.g.…