Decoy-Calibrated Failure Audits for Language Models

Vyzantinos Repantis, Ameya Gawde, Harshvardhan Singh

Useful audits reveal not only how often a model fails, but also where its failures concentrate. An auditor may test many candidate explanations: long inputs, indirect questions, di…