Neural Program Repair with Execution-based Backpropagation
arXiv:2105.04123 · doi:10.1145/3510003.3510222
Abstract
Neural machine translation (NMT) architectures have achieved promising results for automatic program repair. Yet, they have the limitation of generating low-quality patches (e.g., not compilable patches). This is because the existing works only optimize a purely syntactic loss function based on characters and tokens without incorporating program-specific information during neural network weight optimization. In this paper, we propose a novel program repair model called RewardRepair. The core novelty of RewardRepair is to improve NMT-based program repair with a loss function based on program compilation and test execution information, rewarding the network to produce patches that compile and that do not overfit. We conduct several experiments to evaluate RewardRepair showing that it is feasible and effective to use compilation and test execution results to optimize the underlying neural repair model. RewardRepair correctly repairs 207 bugs over four benchmarks. we report on repair success for 121 bugs that are fixed for the first time in the literature. Also, RewardRepair produces up to 45.3% of compilable patches, an improvement over the 39% by the state-of-the-art.
References in corpus (24)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Mixed Precision Training
- Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs
- SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair
- Automatic Software Repair: a Bibliography
- TBar: Revisiting Template-based Automated Program Repair
- CURE: Code-Aware Neural Machine Translation for Automatic Program Repair
- Automatic Repair of Real Bugs in Java: A Large-Scale Experiment on the Defects4J Dataset
- Elixir: Effective object-oriented program repair
- Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code
- On the Efficiency of Test Suite based Program Repair: A Systematic Assessment of 16 Automated Repair Systems for Java Programs
- CODIT: Code Editing with Tree-Based Neural Models
- Bears: An Extensible Java Bug Benchmark for Automatic Program Repair Studies
- On the Generalizability of Neural Program Models with respect to Semantic-Preserving Program Transformations
- Automated Classification of Overfitting Patches with Statically Extracted Code Features
- Automated Patch Assessment for Program Repair at Scale
- Aligned Cross Entropy for Non-Autoregressive Machine Translation
- A Comprehensive Study of Automatic Program Repair on the QuixBugs Benchmark
- A Literature Study of Embeddings on Source Code
- Dynamic Neural Program Embedding for Program Repair
- A Software-Repair Robot based on Continual Learning
- The CodRep Machine Learning on Source Code Competition
- Adaptive Loss Scaling for Mixed Precision Training
Cited by in corpus (23)
- Practical Program Repair in the Era of Large Pre-trained Language Models
- Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review
- Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair
- How Effective Are Neural Networks for Fixing Security Vulnerabilities
- Deep Learning-based Software Engineering: Progress, Challenges, and Opportunities
- SelfAPR: Self-supervised Program Repair with Test Execution Diagnostics
- ITER: Iterative Neural Repair for Multi-Location Patches
- Revisiting the Plastic Surgery Hypothesis via Large Language Models
- Invalidator: Automated Patch Correctness Assessment via Semantic and Syntactic Reasoning
- Automated Test Case Repair Using Language Models
- PyTy: Repairing Static Type Errors in Python
- Is this Change the Answer to that Problem? Correlating Descriptions of Bug and Code Changes for Evaluating Patch Correctness
- The EarlyBIRD Catches the Bug: On Exploiting Early Layers of Encoder Models for More Efficient Code Classification
- DrPlanner: Diagnosis and Repair of Motion Planners for Automated Vehicles Using Large Language Models
- T5APR: Empowering Automated Program Repair across Languages through Checkpoint Ensemble
- Self-Supervised Learning to Prove Equivalence Between Straight-Line Programs via Rewrite Rules
- Practical Program Repair via Preference-based Ensemble Strategy
- STEAM: Simulating the InTeractive BEhavior of ProgrAMmers for Automatic Bug Fixing
- Energy Consumption of Automated Program Repair
- Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixing
- Show Me Why It's Correct: Saving 1/3 of Debugging Time in Program Repair with Interactive Runtime Comparison
- MultiMend: Multilingual Program Repair with Context Augmentation and Multi-Hunk Patch Generation
- Out of Context: How important is Local Context in Neural Program Repair?