A Systematic Literature Review of the Use of GenAI Assistants for Code Comprehension: Implications for Computing Education Research and Practice
arXiv:2510.17894 · doi:10.1145/3785366
Abstract
The ability to comprehend code has long been recognized as an essential skill in software engineering. As programmers lean more heavily on generative artificial intelligence (GenAI) assistants to develop code solutions, it is becoming increasingly important for programmers to comprehend GenAI solutions so that they can verify their appropriateness and properly integrate them into existing code. At the same time, GenAI tools are increasingly being enlisted to provide programmers with tailored explanations of code written both by GenAI and humans. Thus, in computing education, GenAI presents new challenges and opportunities for learners who are trying to comprehend computer programs. To provide computing educators with evidence-based guidance on the use of GenAI to facilitate code comprehension and to identify directions for future research, we present a systematic literature review (SLR) of state-of-the-art approaches and tools that leverage GenAI to enhance code comprehension. Our SLR focuses on 31 studies published between 2022 and 2024. Despite their potential, GenAI assistants often yield inaccurate or unclear explanations, and novice programmers frequently struggle to craft effective prompts, thereby impeding their ability to leverage GenAI to aid code comprehension. Our review classifies GenAI-based approaches and tools, identifies methods used to study them, and summarizes the empirical evaluations of their effectiveness. We consider the implications of our findings for computing education research and practice, and identify directions for future research.
References in corpus (15)
- Automatic Generation of Programming Exercises and Code Explanations using Large Language Models
- The Robots are Here: Navigating the Generative AI Revolution in Computing Education
- Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming
- The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development
- CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs
- Using Large Language Models to Enhance Programming Error Messages
- Comparing Code Explanations Created by Students and Large Language Models
- A Quantitative and Qualitative Evaluation of LLM-Based Explainable Fault Localization
- Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks
- How Readable is Model-generated Code? Examining Readability and Visual Inspection of GitHub Copilot
- Ivie: Lightweight Anchored Explanations of Just-Generated Code
- 40 Years of Designing Code Comprehension Experiments: A Systematic Mapping Study
- The Effects of GitHub Copilot on Computing Students' Programming Effectiveness, Efficiency, and Processes in Brownfield Programming Tasks
- Not the Silver Bullet: LLM-enhanced Programming Error Messages are Ineffective in Practice
- Let's Ask AI About Their Programs: Exploring ChatGPT's Answers To Program Comprehension Questions