Efficient Safety Benchmarking via Item Response Theory

Fabio Spagliardi, Mírian Silva, Ayan Datta +3

Safety benchmarks for language models are typically evaluated using static paradigms that treat all items as equally informative for all models, an assumption that is particularly…