9 citations · 16 across the 11 of their papers we have counts for
5 papers · 2 filters
Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures
Harsh Raj, David Lee, Anas Mahmoud +7
The increasing deployment of AI agents in long-horizon tasks yields massive execution logs. Diagnosing failures within these records is crucial for reliability, as it transforms ou…
Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
Lin Shi, Haowei Lin, Zixuan Zhu +123
Evaluating agents on the growing number of agentic benchmarks is challenging because they often require complex environments and agent integrations. We introduce Harbor Adapters, a…
Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures
Harsh Raj, Vipul Gupta, Anas Mahmoud +4
Existing evaluations often reduce agent failures to system-level outcomes, obscuring where the fault originated and which intervention would improve the agent system. This creates…
OpenThoughts-Agent: Data Recipes for Agentic Models
Negin Raoof, Richard Zhuang, Marianna Nezhurina +47
Agentic language models dramatically expand the applications of AI yet little is publicly known about how to curate training data for broadly capable agents. Existing open efforts…
Consistency as a Testable Property: Statistical Methods to Evaluate AI Agent Reliability
Harsh Raj, Niranjan Orkat, Suvrorup Mukherjee +3
This paper establishes a rigorous measurement science for AI agent reliability, providing a foundational framework for quantifying consistency under semantically preserving perturb…