Item Response Theory for AI Safety

Joshua Fonseca Rivera, Neil Shah, David Demitri Africa +1

Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to trust and interpret, because b…