works on
benchmark evaluation 1item response theory 1large language models 1model ranking 1simulation study 1

From the 1 of 1 linked paper with an AI index.

collaborators

1 paper

cs.AI2026

Can We Trust Item Response Theory for AI Evaluation?

Han Jiang, Sunbeom Kwon, Jinwen Luo +2

The paper investigates how well item response theory (IRT) works for evaluating large language model benchmarks, highlighting challenges when benchmark data differ from traditional…