1 citations · 1 across the 4 of their papers we have counts for
4 papers · 1 filter
Alice: A Large-Scale German Benchmark for Rubric-Based Multi-Dimensional Automatic Short Answer Scoring
Zhifan Sun, Sebastian Gombert, Jannik Lossjew +6
Automatic Short Answer Scoring (ASAS) is central to NLP for Education. However, openly available benchmarks remain scarce, and existing datasets largely address how well students a…
Evaluating LLM-as-a-Judge Beyond Score Alignment: A Psychometric Analysis of Residual Judging Difficulty
Longwei Cong, Sonja Hahn, Sebastian Gombert +4
Large language models (LLMs) are widely used as automatic judges, with validity typically assessed via alignment with human scores. However, aggregate agreement fails to reveal whe…
Estimating LLM Grading Ability and Response Difficulty in Automatic Short Answer Grading via Item Response Theory
Longwei Cong, Sonja Hahn, Sebastian Gombert +3
Automated short answer grading (ASAG) with large language models (LLMs) is commonly evaluated with aggregate metrics such as macro-F1 and Cohen's kappa. However, these metrics prov…
Confidence Estimation in Automatic Short Answer Grading with LLMs
Longwei Cong, Sonja Hahn, Sebastian Gombert +3
Automatic Short Answer Grading (ASAG) with generative large language models (LLMs) has recently demonstrated strong performance without task-specific fine-tuning, while also enabli…