473 citations · 1.1k across the 16 of their papers we have counts for
16 papers
LLM-itation is the Sincerest Form of Data: Generating Synthetic Buggy Code Submissions for Computing Education
Juho Leinonen, Paul Denny, Olli Kiljunen +3
There is a great need for data in computing education research. Data is needed to understand how students behave, to train models of student behavior to optimally support students,…
Evaluating Language Models for Generating and Judging Programming Feedback
Charles Koutcheme, Nicola Dainese, Arto Hellas +4
The emergence of large language models (LLMs) has transformed research and practice across a wide range of domains. Within the computing education research (CER) domain, LLMs have…
Evaluating Contextually Personalized Programming Exercises Created with Generative AI
Evanfiya Logacheva, Arto Hellas, James Prather +2
Programming skills are typically developed through completing various hands-on exercises. Such programming problems can be contextualized to students' interests and cultural backgr…
Benchmarking Educational Program Repair
Charles Koutcheme, Nicola Dainese, Sami Sarsa +3
The emergence of large language models (LLMs) has sparked enormous interest due to their potential application across a range of educational tasks. For example, recent work in prog…
Open Source Language Models Can Provide Feedback: Evaluating LLMs' Ability to Help Students Using GPT-4-As-A-Judge
Charles Koutcheme, Nicola Dainese, Sami Sarsa +3
Large language models (LLMs) have shown great potential for the automatic generation of feedback in a wide range of computing contexts. However, concerns have been voiced around th…
Decoding Logic Errors: A Comparative Study on Bug Detection by Students and Large Language Models
Stephen MacNeil, Paul Denny, Andrew Tran +5
Identifying and resolving logic errors can be one of the most frustrating challenges for novices programmers. Unlike syntax errors, for which a compiler or interpreter can issue a…
Can We Trust AI-Generated Educational Content? Comparative Analysis of Human and AI-Generated Learning Resources
Paul Denny, Hassan Khosravi, Arto Hellas +2
As an increasing number of students move to online learning platforms that deliver personalized learning experiences, there is a great need for the production of high-quality educa…
Exploring the Responses of Large Language Models to Beginner Programmers' Help Requests
Arto Hellas, Juho Leinonen, Sami Sarsa +3
Background and Context: Over the past year, large language models (LLMs) have taken the world by storm. In computing education, like in other walks of life, many opportunities and…
Computing Education in the Era of Generative AI
Paul Denny, James Prather, Brett A. Becker +7
The computing education community has a rich history of pedagogical innovation designed to support students in introductory courses, and to support teachers in facilitating student…
Comparing Code Explanations Created by Students and Large Language Models
Juho Leinonen, Paul Denny, Stephen MacNeil +5
Reasoning about code and explaining its purpose are fundamental skills for computer scientists. There has been extensive research in the field of computing education on the relatio…
Automatically Generating CS Learning Materials with Large Language Models
Stephen MacNeil, Andrew Tran, Juho Leinonen +5
Recent breakthroughs in Large Language Models (LLMs), such as GPT-3 and Codex, now enable software developers to generate code based on a natural language prompt. Within computer s…
Robosourcing Educational Resources -- Leveraging Large Language Models for Learnersourcing
Paul Denny, Sami Sarsa, Arto Hellas +1
In this article, we introduce and evaluate the concept of robosourcing for creating educational content. Robosourcing lies in the intersection of crowdsourcing and large language m…
Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book
Stephen MacNeil, Andrew Tran, Arto Hellas +5
Advances in natural language processing have resulted in large language models (LLMs) that are capable of generating understandable and sensible written text. Recent versions of th…
Using Large Language Models to Enhance Programming Error Messages
Juho Leinonen, Arto Hellas, Sami Sarsa +4
A key part of learning to program is learning to understand programming error messages. They can be hard to interpret and identifying the cause of errors can be time-consuming. One…
Automatic Generation of Programming Exercises and Code Explanations using Large Language Models
Sami Sarsa, Paul Denny, Arto Hellas +1
This article explores the natural language generation capabilities of large language models with application to the production of two types of learning resources common in programm…
Empirical Evaluation of Deep Learning Models for Knowledge Tracing: Of Hyperparameters and Metrics on Performance and Replicability
Sami Sarsa, Juho Leinonen, Arto Hellas
We review and evaluate a body of deep learning knowledge tracing (DLKT) models with openly available and widely-used data sets, and with a novel data set of students learning to pr…