Unlimited Practice Opportunities: Automated Generation of Comprehensive, Personalized Programming Tasks
arXiv:2503.11704 · doi:10.1145/3724363.3729089
Abstract
Generative artificial intelligence (GenAI) offers new possibilities for generating personalized programming exercises, addressing the need for individual practice. However, the task quality along with the student perspective on such generated tasks remains largely unexplored. Therefore, this paper introduces and evaluates a new feature of the so-called Tutor Kai for generating comprehensive programming tasks, including problem descriptions, code skeletons, unit tests, and model solutions. The presented system allows students to freely choose programming concepts and contextual themes for their tasks. To evaluate the system, we conducted a two-phase mixed-methods study comprising (1) an expert rating of 200 automatically generated programming tasks w.r.t. task quality, and (2) a study with 26 computer science students who solved and rated the personalized programming tasks. Results show that experts classified 89.5% of the generated tasks as functional and 92.5% as solvable. However, the system's rate for implementing all requested programming concepts decreased from 94% for single-concept tasks to 40% for tasks addressing three concepts. The student evaluation further revealed high satisfaction with the personalization. Students also reported perceived benefits for learning. The results imply that the new feature has the potential to offer students individual tasks aligned with their context and need for exercise. Tool developers, educators, and, above all, students can benefit from these insights and the system itself.
Accepted for ITiCSE'25
References in corpus (11)
- Automatic Generation of Programming Exercises and Code Explanations using Large Language Models
- The Robots are Here: Navigating the Generative AI Revolution in Computing Education
- Prompt Injection attack against LLM-integrated Applications
- Feedback-Generation for Programming Exercises With GPT-4
- Evaluating the Application of Large Language Models to Generate Feedback in Programming Education
- Can We Trust AI-Generated Educational Content? Comparative Analysis of Human and AI-Generated Learning Resources
- Automating Autograding: Large Language Models as Test Suite Generators for Introductory Programming
- Robosourcing Educational Resources -- Leveraging Large Language Models for Learnersourcing
- Leveraging Lecture Content for Improved Feedback: Explorations with GPT-4 and Retrieval Augmented Generation
- Automating Personalized Parsons Problems with Customized Contexts and Concepts
- Neural Task Synthesis for Visual Programming