Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-shot Learning
arXiv:2207.08281 · doi:10.1145/3540250.3549101
Abstract
Due to the promising future of Automated Program Repair (APR), researchers have proposed various APR techniques, including heuristic-based, template-based, and constraint-based techniques. Among such classic APR techniques, template-based techniques have been widely recognized as state of the art. However, such template-based techniques require predefined templates to perform repair, and their effectiveness is thus limited. To this end, researchers leveraged the recent advances in Deep Learning to further improve APR. Such learning-based techniques view APR as a Neural Machine Translation problem, using the buggy/fixed code snippets as the source/target languages for translation. In this way, such techniques heavily rely on large numbers of high-quality bug-fixing commits, which can be extremely costly and challenging to construct. Furthermore, the edit variety of these learning-based techniques are limited to the available bug-fixes within their training datasets. Therefore, in this paper, we aim to revisit the learning-based APR problem, and propose AlphaRepair, to leverage zero-shot learning directly using large pre-trained code models for APR. Our main insight is instead of modeling what a repair edit should look like, we can directly predict what the correct code is based on the context information. We have implemented AlphaRepair as a practical multilingual APR tool based on the recent CodeBERT model. Our results on the widely used Defects4J benchmark show that AlphaRepair can substantially outperform state-of-the-art APR tools. We also studied the impact of different design choices and show that AlphaRepair performs even better on a newer version of Defects4J (2.0) with 3.3X more fixes than best performing baseline, indicating that AlphaRepair can potentially avoid the dataset-overfitting issue of existing learning-based techniques.
Accepted at ESEC/FSE 2022
References in corpus (4)
Cited by in corpus (25)
- Practical Program Repair in the Era of Large Pre-trained Language Models
- Keep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT
- Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review
- Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code
- Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair
- The Future of AI-Driven Software Engineering
- ITER: Iterative Neural Repair for Multi-Location Patches
- Fuzzing Automatic Differentiation in Deep-Learning Libraries
- Harnessing the Power of LLM to Support Binary Taint Analysis
- Revisiting the Plastic Surgery Hypothesis via Large Language Models
- ContrastRepair: Enhancing Conversation-Based Automated Program Repair via Contrastive Test Case Pairs
- RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair
- FlakyFix: Using Large Language Models for Predicting Flaky Test Fix Categories and Test Code Repair
- Automated Test Case Repair Using Language Models
- PyTy: Repairing Static Type Errors in Python
- DrPlanner: Diagnosis and Repair of Motion Planners for Automated Vehicles Using Large Language Models
- Practical Program Repair via Preference-based Ensemble Strategy
- RepairBench: Leaderboard of Frontier Models for Program Repair
- STEAM: Simulating the InTeractive BEhavior of ProgrAMmers for Automatic Bug Fixing
- Program Decomposition and Translation with Static Analysis
- Risks of ignoring uncertainty propagation in AI-augmented security pipelines
- Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixing
- Assessing the Latent Automated Program Repair Capabilities of Large Language Models using Round-Trip Translation
- MultiMend: Multilingual Program Repair with Context Augmentation and Multi-Hunk Patch Generation
- Out of Context: How important is Local Context in Neural Program Repair?