Auditing Failure Attribution: Label Predictability and Judge Behavior

Tianyang Xu, Jing Gao

Failure-attribution benchmarks ask an LLM judge which step or agent caused a failure. A high score should mean the judge checked the faulty step against the run (a cross-step check…