9 papers · 1 filter
Analyzing the Difficulty of Programming Assignments with Interpretable Knowledge Component Metrics
Tsvetomila Mihaylova, Jing Fan, Bita Akram +4
This research paper examines how Knowledge Components (KCs) - fine-grained concepts or skills required to solve programming tasks - can be used as interpretable signals for underst…
The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness
Rose Niousha, Samantha Boatright Smith, Bita Akram +5
Current Artificial Intelligence (AI)-based tutoring systems (AI tutors) are primarily evaluated based on the pedagogical quality of their feedback messages. While important, pedago…
How Large Language Models Are Changing MOOC Essay Answers: A Comparison of Pre- and Post-LLM Responses
Leo Leppänen, Lili Aunimo, Arto Hellas +2
The release of ChatGPT in late 2022 caused a flurry of activity and concern in the academic and educational communities. Some see the tool's ability to generate human-like text tha…
LLM-itation is the Sincerest Form of Data: Generating Synthetic Buggy Code Submissions for Computing Education
Juho Leinonen, Paul Denny, Olli Kiljunen +3
There is a great need for data in computing education research. Data is needed to understand how students behave, to train models of student behavior to optimally support students,…
On the Opportunities of Large Language Models for Programming Process Data
John Edwards, Arto Hellas, Juho Leinonen
Computing educators and researchers have used programming process data to understand how programs are constructed and what sorts of problems students struggle with. Although such d…
Comparing the Utility, Preference, and Performance of Course Material Search Functionality and Retrieval-Augmented Generation Large Language Model (RAG-LLM) AI Chatbots in Information-Seeking Tasks
Leonardo Pasquarelli, Charles Koutcheme, Arto Hellas
Providing sufficient support for students requires substantial resources, especially considering the growing enrollment numbers. Students need help in a variety of tasks, ranging f…