6 papers · 1 filter
Beyond Agreement: Rethinking Ground Truth in Educational AI Annotation
Danielle R. Thomas, Conrad Borchers, Kenneth R. Koedinger
Humans can be notoriously imperfect evaluators. They are often biased, unreliable, and unfit to define "ground truth." Yet, given the surging need to produce large amounts of train…
Leveraging LLMs to Assess Tutor Moves in Real-Life Dialogues: A Feasibility Study
Danielle R. Thomas, Conrad Borchers, Jionghao Lin +6
Tutoring improves student achievement, but identifying and studying what tutoring actions are most associated with student learning at scale based on audio transcriptions is an ope…
Detecting LLM-Generated Short Answers and Effects on Learner Performance
Shambhavi Bhushan, Danielle R Thomas, Conrad Borchers +5
The increasing availability of large language models (LLMs) has raised concerns about their potential misuse in online learning. While tools for detecting LLM-generated text exist…
LLM-Generated Feedback Supports Learning If Learners Choose to Use It
Danielle R. Thomas, Conrad Borchers, Shambhavi Bhushan +3
Large language models (LLMs) are increasingly used to generate feedback, yet their impact on learning remains underexplored, especially compared to existing feedback methods. This…
Augmenting Human-Annotated Training Data with Large Language Model Generation and Distillation in Open-Response Assessment
Conrad Borchers, Danielle R. Thomas, Jionghao Lin +2
Large Language Models (LLMs) like GPT-4o can help automate text classification tasks at low cost and scale. However, there are major concerns about the validity and reliability of…
Comparing Few-Shot Prompting of GPT-4 LLMs with BERT Classifiers for Open-Response Assessment in Tutor Equity Training
Sanjit Kakarla, Conrad Borchers, Danielle Thomas +2
Assessing learners in ill-defined domains, such as scenario-based human tutoring training, is an area of limited research. Equity training requires a nuanced understanding of conte…