5 papers
MortarBench: Evaluating Mortgage Loan Origination Agents
Matthew Toles, Yunan Lu, Manav Munjal +6
Loan origination is the process by which a lender creates a new loan, from application and underwriting through approval and funding. This process serves a critical role in evaluat…
EARS: Explanatory Abstention for Reliable Sub-Agent Modeling in Large-scale Multi-Agent Systems
Shuang Xie, Yunan Lu, Han Li +1
In large-scale enterprise settings, centralized multi-agent systems (MAS) are increasingly adopted, in which a coordinator delegates user requests to lightweight, domain-specialize…
VISTA: A Versatile Interactive User Simulation Toolkit for Agent Evaluation
Yunan Lu, Ryan Shea, Yusen Zhang +1
Evaluation remains a critical bottleneck for interactive agent development. Existing evaluation methods often rely on static benchmarks, which fail to capture the dynamic, multi-st…
Speculative Actions: A Lossless Framework for Faster Agentic Systems
Naimeng Ye, Arnav Ahuja, Georgios Liargkovas +3
AI agents are increasingly deployed in complex, interactive environments, yet their runtime remains a major bottleneck for training, evaluation, and real-world use. Typical agent b…
SAGE: A Top-Down Bottom-Up Knowledge-Grounded User Simulator for Multi-turn AGent Evaluation
Ryan Shea, Yunan Lu, Liang Qiu +1
Evaluating multi-turn interactive agents is challenging due to the need for human assessment. Evaluation with simulated users has been introduced as an alternative, however existin…