JE-IRT: A Geometric Lens on LLM Abilities through Joint Embedding Item Response Theory

Louie Hong Yao, Nicholas Jarvis, Tiffany Zhan +3

Standard LLM evaluation practices compress diverse abilities into single scores, obscuring their inherently multidimensional nature. We present JE-IRT, a geometric item-response fr…