3 papers
cs.AI2026
A Hierarchical Statistical Framework for Evaluating Agent Reliability
Mirco Milletarì, Tuhin Kundu
Agents operating in complex scenarios often face uncertainty due to incomplete information about the environment they operate in, ambiguous user queries, complex business rules and…
cs.AI2026
Towards an AI Software Factory for Data Systems
Anna Pavlenko, Bogdan Crivat, Brandon Haynes +23
AI-assisted coding tools deliver significant acceleration of coding, but only limited impact across the end-to-end software development lifecycle (SDLC)--an Amdahl's law effect! In…
cs.CL2026
One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows
Zhuochun Li, Youngmin Ko, Ali Keramati +12
Recent agent benchmarks increasingly ground evaluation in executable environments, from code repair to web navigation, app APIs, and function calling. Yet completing consequential…