26 citations · 26 across the 2 of their papers we have counts for
4 papers
SkillEval: Learning Interpretable Ability Profiles of LLMs via Cognitive Diagnosis Models
Berke Arda, Peng Cui, Qiaoyuan Zheng +3
Current evaluation of LLMs on benchmarks typically aggregates performance over individual examples into a single scalar score. This approach is overly coarse and ignores the divers…
What Makes Something Hard(er)? Explaining Question Difficulty in Natural Language
Peng Cui, Qiaoyuan Zheng, Rudolf Debelak +1
Difficulty is one of the most fundamental properties of a question: it determines whether the question can meaningfully discriminate between models of differing ability. Although a…
FairTree: Subgroup Fairness Auditing of Machine Learning Models with Bias-Variance Decomposition
Rudolf Debelak
The evaluation of machine learning models typically relies mainly on performance metrics based on loss functions, which risk to overlook changes in performance in relevant subgroup…
Model Selection of Nested and Non-Nested Item Response Models using Vuong Tests
Lennart Schneider, R. Philip Chalmers, Rudolf Debelak +1
In this paper, we apply Vuong's (1989) general approach of model selection to the comparison of nested and non-nested unidimensional and multidimensional item response theory (IRT)…