Measurement Under Selection: Decoy-Calibrated Failure Audits for Language Models

Vyzantinos Repantis, Ameya Gawde, Harshvardhan Singh

Knowing how often a language model fails does not explain where its errors concentrate. When auditors examine many explanations, the strongest observed pattern may arise by chance.…