1 citations · 1 across the 4 of their papers we have counts for
5 papers
Are Benchmarks Reliable? Toward Structural Diagnosis via Item-Level Capability Boundaries
Haiquan Hu, Yuzhu Liang, Weicheng Tang +4
Evaluating large language models (LLMs) relies heavily on benchmark scores, yet aggregate metrics can obscure whether individual items serve their intended measurement purpose. BSD…
BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data
Jaap Jumelet, Abdellah Fourtassi, Akari Haga +23
We present BabyBabelLM, a multilingual collection of datasets modeling the language a person observes from birth until they acquire a native language. We curate developmentally pla…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
STEM: Efficient Relative Capability Evaluation of LLMs through Structured Transition Samples
Haiquan Hu, Jiazhi Jiang, Shiyou Xu +2
Evaluating large language models (LLMs) has become increasingly challenging as model capabilities advance rapidly. While recent models often achieve higher scores on standard bench…
Knowing When to Critique: Task-Adaptive Metacognitive Regulation for Reliable LLM Reasoning
Xinmeng Hou, Ziting Chang, Zhouquan Lu +6
Large language models (LLMs) reason fluently but do not regulate their reasoning: they apply uniform scrutiny to every input, which leaves them vulnerable to adversarial and counte…