LLM Evaluation on Unseen Questions: Contextual Multidimensional IRT Model

Ergan Shang, Weijing Tang, Yinqiu He

Evaluation of large language models (LLMs) increasingly requires predicting how a model will perform on new questions or tasks before collecting large amounts of new annotations. T…