473 citations · 1.2k across the 39 of their papers we have counts for
41 papers
Pulla: A Parsons Problem Tool for Fine-Grained Behavioral Tracing and Instructor-Facing Problem-Solving Analysis
Daniel Prol, Juho Leinonen, Arto Hellas +2
Existing Parsons problem tools primarily focus on correctness, indicating whether a student solved a problem, but providing limited visibility into the underlying problem-solving p…
Analyzing the Difficulty of Programming Assignments with Interpretable Knowledge Component Metrics
Tsvetomila Mihaylova, Jing Fan, Bita Akram +4
This research paper examines how Knowledge Components (KCs) - fine-grained concepts or skills required to solve programming tasks - can be used as interpretable signals for underst…
The Effects of Structured LLM-Generated Feedback on Programming Assignment Performance
Tsvetomila Mihaylova, Evanfiya Logacheva, Arto Hellas +6
When programming students encounter errors in their code, compiler messages or static analysis output often provide limited guidance, particularly for novice programmers. Personali…
AI-Generated Slides: Are They Good? Can Students Tell?
Juho Leinonen, Lisa Zhang, Arto Hellas
As generative AI (GenAI) tools become easily accessible, there is promise in using such tools to support instructors. To that end, this paper examines using GenAI to help generate…
Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education
Mragisha Jain, Tirth Bhatt, Griffin Pitts +6
Students learning algorithms often need support as they interpret traces, debug reasoning errors, and apply procedures across unfamiliar problem instances. In this paper, we presen…
Teaching Language Models How to Code Like Learners: Conversational Serialization for Student Simulation
Charles Koutcheme, Juho Leinonen, Arto Hellas
Artificial students -- models that simulate how learners act and respond within educational systems -- are a promising tool for evaluating tutoring strategies and feedback mechanis…
The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness
Rose Niousha, Samantha Boatright Smith, Bita Akram +5
Current Artificial Intelligence (AI)-based tutoring systems (AI tutors) are primarily evaluated based on the pedagogical quality of their feedback messages. While important, pedago…
Personalized Worked Example Generation from Student Code Submissions Using Pattern-based Knowledge Components
Griffin Pitts, Muntasir Hoq, Peter Brusilovsky +4
Adaptive programming practice often relies on fixed libraries of worked examples and practice problems, which require substantial authoring effort and may not correspond well to th…
When Looks Do Not Lie: Discourse Structure Guided In-Context Learning for Faithful Diagram Generation
Evanfiya Logacheva, Arto Hellas, Tsvetomila Mihaylova +3
GenAI is widespread in educational applications; however, it is known to generate content with intrinsic and extrinsic hallucination. We introduce a novel method for ICL diagram ge…
NEFMind: Parameter-Efficient Fine-Tuning of Open-Source LLMs for Telecom APIs Automation
Zainab Khan, Ahmed Hussain, Mukesh Thakur +2
The use of Service-Based Architecture in modern telecommunications has exponentially increased Network Functions (NFs) and Application Programming Interfaces (APIs), creating subst…
How Large Language Models Are Changing MOOC Essay Answers: A Comparison of Pre- and Post-LLM Responses
Leo Leppänen, Lili Aunimo, Arto Hellas +2
The release of ChatGPT in late 2022 caused a flurry of activity and concern in the academic and educational communities. Some see the tool's ability to generate human-like text tha…
Two Approaches for Programming Education in the Domain of Graphics: An Experiment
Luca Chiodini, Juha Sorva, Arto Hellas +2
Context: Graphics is a popular domain for teaching introductory programming in a motivating way, even in text-based programming languages. Over the last few decades, a large number…
Overview of Web Application Performance Optimization Techniques
Juho Vepsäläinen, Arto Hellas, Petri Vuorimaa
During its thirty years of existence, the World Wide Web has helped to transform the world and create digital economies. Although it started as a global information exchange, it ha…
LLM-itation is the Sincerest Form of Data: Generating Synthetic Buggy Code Submissions for Computing Education
Juho Leinonen, Paul Denny, Olli Kiljunen +3
There is a great need for data in computing education research. Data is needed to understand how students behave, to train models of student behavior to optimally support students,…
On the Opportunities of Large Language Models for Programming Process Data
John Edwards, Arto Hellas, Juho Leinonen
Computing educators and researchers have used programming process data to understand how programs are constructed and what sorts of problems students struggle with. Although such d…
Comparing the Utility, Preference, and Performance of Course Material Search Functionality and Retrieval-Augmented Generation Large Language Model (RAG-LLM) AI Chatbots in Information-Seeking Tasks
Leonardo Pasquarelli, Charles Koutcheme, Arto Hellas
Providing sufficient support for students requires substantial resources, especially considering the growing enrollment numbers. Students need help in a variety of tasks, ranging f…
Synthetic Students: A Comparative Study of Bug Distribution Between Large Language Models and Computing Students
Stephen MacNeil, Magdalena Rogalska, Juho Leinonen +3
Large language models (LLMs) present an exciting opportunity for generating synthetic classroom data. Such data could include code containing a typical distribution of errors, simu…
Evaluating Language Models for Generating and Judging Programming Feedback
Charles Koutcheme, Nicola Dainese, Arto Hellas +4
The emergence of large language models (LLMs) has transformed research and practice across a wide range of domains. Within the computing education research (CER) domain, LLMs have…
Evaluating Contextually Personalized Programming Exercises Created with Generative AI
Evanfiya Logacheva, Arto Hellas, James Prather +2
Programming skills are typically developed through completing various hands-on exercises. Such programming problems can be contextualized to students' interests and cultural backgr…
Experiences from Integrating Large Language Model Chatbots into the Classroom
Arto Hellas, Juho Leinonen, Leo Leppänen
In the present study, we provided students an unfiltered access to a state-of-the-art large language model (LLM) chatbot. The chatbot was intentionally designed to mimic proprietar…
Benchmarking Educational Program Repair
Charles Koutcheme, Nicola Dainese, Sami Sarsa +3
The emergence of large language models (LLMs) has sparked enormous interest due to their potential application across a range of educational tasks. For example, recent work in prog…
Open Source Language Models Can Provide Feedback: Evaluating LLMs' Ability to Help Students Using GPT-4-As-A-Judge
Charles Koutcheme, Nicola Dainese, Sami Sarsa +3
Large language models (LLMs) have shown great potential for the automatic generation of feedback in a wide range of computing contexts. However, concerns have been voiced around th…
Let's Ask AI About Their Programs: Exploring ChatGPT's Answers To Program Comprehension Questions
Teemu Lehtinen, Charles Koutcheme, Arto Hellas
Recent research has explored the creation of questions from code submitted by students. These Questions about Learners' Code (QLCs) are created through program analysis, exploring…
"Like a Nesting Doll": Analyzing Recursion Analogies Generated by CS Students using Large Language Models
Seth Bernstein, Paul Denny, Juho Leinonen +5
Grasping complex computing concepts often poses a challenge for students who struggle to anchor these new ideas to familiar experiences and understandings. To help with this, a goo…