Evaluating the Application of Large Language Models to Generate Feedback in Programming Education
arXiv:2403.09744 · doi:10.1109/EDUCON60312.2024.10578838
Abstract
This study investigates the application of large language models, specifically GPT-4, to enhance programming education. The research outlines the design of a web application that uses GPT-4 to provide feedback on programming tasks, without giving away the solution. A web application for working on programming tasks was developed for the study and evaluated with 51 students over the course of one semester. The results show that most of the feedback generated by GPT-4 effectively addressed code errors. However, challenges with incorrect suggestions and hallucinated issues indicate the need for further improvements.
accepted at IEEE Global Engineering Education Conference 2024, Kos, Greece
References in corpus (5)
- The Robots are Here: Navigating the Generative AI Revolution in Computing Education
- CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs
- Exploring the Responses of Large Language Models to Beginner Programmers' Help Requests
- Feedback-Generation for Programming Exercises With GPT-4
- AI-enhanced Auto-correction of Programming Exercises: How Effective is GPT-3.5?
Cited by in corpus (7)
- Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubrics
- The Responsible Development of Automated Student Feedback with Generative AI
- Leveraging Lecture Content for Improved Feedback: Explorations with GPT-4 and Retrieval Augmented Generation
- That's Not the Feedback I Need! -- Student Engagement with GenAI Feedback in the Tutor Kai
- Unlimited Practice Opportunities: Automated Generation of Comprehensive, Personalized Programming Tasks
- GenAI Voice Mode in Programming Education
- Student Engagement with GenAI's Tutoring Feedback: A Mixed Methods Study