What Counts as the Same Task? Executable Contracts for Robustness Evaluation of Tool-Using Agents

Shaikh Akib Shahriyar, MD. Akil Raihan Iftee, Armon Barton +1

Robustness tests for tool-using agents often change the agent's environment, for example by hiding a hostile note in an invoice, and then judge the agent by what it does. The resul…