10 citations · 15 across the 6 of their papers we have counts for
7 papers
VAKRA: Evaluating Multi-Hop Reasoning Across APIs and Retrieval Under Tool-Use Policies
Ankita Rajaram Naik, Anupama Murthi, Benjamin Elder +6
Agents deployed in enterprise settings must reason across structured APIs and document collections, yet existing benchmarks evaluate these capabilities in isolation. We introduce V…
Agent Lifecycle Toolkit (ALTK): Reusable Middleware Components for Robust AI Agents
Zidane Wright, Jason Tsay, Anupama Murthi +8
As AI agents move from demos into enterprise deployments, their failure modes become consequential: a misinterpreted tool argument can corrupt production data, a silent reasoning e…
Live API-Bench: 2500+ Live APIs for Testing Multi-Step Tool Calling
Benjamin Elder, Anupama Murthi, Jungkoo Kang +4
Large language models (LLMs) increasingly rely on external tools and APIs to execute complex tasks specified in natural language. Evaluating such tool calling capabilities in reali…
Learning Prediction Intervals for Model Performance
Benjamin Elder, Matthew Arnold, Anupama Murthi +1
Understanding model performance on unlabeled data is a fundamental challenge of developing, deploying, and maintaining AI systems. Model performance is typically evaluated using te…
Not Your Grandfathers Test Set: Reducing Labeling Effort for Testing
Begum Taskazan, Jiri Navratil, Matthew Arnold +3
Building and maintaining high-quality test sets remains a laborious and expensive task. As a result, test sets in the real world are often not properly kept up to date and drift fr…
Towards Automating the AI Operations Lifecycle
Matthew Arnold, Jeffrey Boston, Michael Desmond +5
Today's AI deployments often require significant human involvement and skill in the operational stages of the model lifecycle, including pre-release testing, monitoring, problem di…
Exploring the Hyperparameter Landscape of Adversarial Robustness
Evelyn Duesterwald, Anupama Murthi, Ganesh Venkataraman +2
Adversarial training shows promise as an approach for training models that are robust towards adversarial perturbation. In this paper, we explore some of the practical challenges o…