1 citations · 1 across the 5 of their papers we have counts for
7 papers
SkillEval: Learning Interpretable Ability Profiles of LLMs via Cognitive Diagnosis Models
Berke Arda, Peng Cui, Qiaoyuan Zheng +3
Current evaluation of LLMs on benchmarks typically aggregates performance over individual examples into a single scalar score. This approach is overly coarse and ignores the divers…
What Makes Something Hard(er)? Explaining Question Difficulty in Natural Language
Peng Cui, Qiaoyuan Zheng, Rudolf Debelak +1
Difficulty is one of the most fundamental properties of a question: it determines whether the question can meaningfully discriminate between models of differing ability. Although a…
Do LLMs Exhibit Coherent Knowledge Structures in Mathematical Reasoning? A Perspective from Knowledge Space Theory
Peng Cui, Heejin Do, Mrinmaya Sachan
Human knowledge is inherently structured and interdependent: mastery of a concept requires prior mastery of its prerequisites, a principle formalized by Knowledge Space Theory (KST…
Grammar Control in Dialogue Response Generation for Language Learning Chatbots
Dominik Glandorf, Peng Cui, Detmar Meurers +1
Chatbots based on large language models offer cheap conversation practice opportunities for language learners. However, they are hard to control for linguistic forms that correspon…
Investigating the Zone of Proximal Development of Language Models for In-Context Learning
Peng Cui, Mrinmaya Sachan
In this paper, we introduce a learning analytics framework to analyze the in-context learning (ICL) behavior of large language models (LLMs) through the lens of the Zone of Proxima…
How to Select Datapoints for Efficient Human Evaluation of NLG Models?
Vilém Zouhar, Peng Cui, Mrinmaya Sachan
Human evaluation is the gold standard for evaluating text generation models. However, it is expensive. In order to fit budgetary constraints, a random subset of the test data is of…
How to Engage Your Readers? Generating Guiding Questions to Promote Active Reading
Peng Cui, Vilém Zouhar, Xiaoyu Zhang +1
Using questions in written text is an effective strategy to enhance readability. However, what makes an active reading question good, what the linguistic role of these questions is…