4 papers
Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degradation and the Impact of Task-Specific Adaptation
Abigail Victoria Gurin Schleifer, Moriah Ariely, Beata Beigman Klebanov +2
Automated short answer scoring (ASAS) is shifting from discriminative, fine-tuned models to large language models (LLMs) used in few-shot settings. This paradigm leverages LLMs bro…
Assessment Design in the AI Era: A Method for Identifying Items Functioning Differentially for Humans and Chatbots
Licol Zeinfeld, Alona Strugatski, Ziva Bar-Dov +3
The rapid adoption of large language models (LLMs) in education raises profound challenges for assessment design. To adapt assessments to the presence of LLM-based tools, it is cru…
Applying IRT to Distinguish Between Human and Generative AI Responses to Multiple-Choice Assessments
Alona Strugatski, Giora Alexandron
Generative AI is transforming the educational landscape, raising significant concerns about cheating. Despite the widespread use of multiple-choice questions in assessments, the de…
Anna Karenina Strikes Again: Pre-Trained LLM Embeddings May Favor High-Performing Learners
Abigail Gurin Schleifer, Beata Beigman Klebanov, Moriah Ariely +1
Unsupervised clustering of student responses to open-ended questions into behavioral and cognitive profiles using pre-trained LLM embeddings is an emerging technique, but little is…