Deep Learning-based Software Engineering: Progress, Challenges, and Opportunities
arXiv:2410.13110 · doi:10.1007/s11432-023-4127-5
Abstract
Researchers have recently achieved significant advances in deep learning techniques, which in turn has substantially advanced other research disciplines, such as natural language processing, image processing, speech recognition, and software engineering. Various deep learning techniques have been successfully employed to facilitate software engineering tasks, including code generation, software refactoring, and fault localization. Many papers have also been presented in top conferences and journals, demonstrating the applications of deep learning techniques in resolving various software engineering tasks. However, although several surveys have provided overall pictures of the application of deep learning techniques in software engineering, they focus more on learning techniques, that is, what kind of deep learning techniques are employed and how deep models are trained or fine-tuned for software engineering tasks. We still lack surveys explaining the advances of subareas in software engineering driven by deep learning techniques, as well as challenges and opportunities in each subarea. To this end, in this paper, we present the first task-oriented survey on deep learning-based software engineering. It covers twelve major software engineering subareas significantly impacted by deep learning techniques. Such subareas spread out the through the whole lifecycle of software development and maintenance, including requirements engineering, software development, testing, maintenance, and developer collaboration. As we believe that deep learning may provide an opportunity to revolutionize the whole discipline of software engineering, providing one survey covering as many subareas as possible in software engineering can help future research push forward the frontier of deep learning-based software engineering more systematically.
Accepted in SCIENCE CHINA Information Sciences
References in corpus (98)
- Distributed Representations of Words and Phrases and their Compositionality
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- BioBERT: a pre-trained biomedical language representation model for biomedical text mining
- Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
- SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair
- TBar: Revisiting Template-based Automated Program Repair
- SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
- Automatic Repair of Real Bugs in Java: A Large-Scale Experiment on the Defects4J Dataset
- CodeBERT: A Pre-Trained Model for Programming and Natural Languages
- Sorting and Transforming Program Repair Ingredients via Deep Learning Code Similarities
- GraphCodeBERT: Pre-training Code Representations with Data Flow
- Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks
- Neural Transfer Learning for Repairing Security Vulnerabilities in C Code
- Neural Program Repair with Execution-based Backpropagation
- Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges
- On the Feasibility of Transfer-learning Code Smells using Deep Learning
- Pythia: AI-assisted Code Completion System
- Large Language Models for Software Engineering: A Systematic Literature Review
- CODIT: Code Editing with Tree-Based Neural Models
- Aroma: Code Recommendation via Structural Code Search
- A Retrieve-and-Edit Framework for Predicting Structured Outputs
- Self-Supervised Contrastive Learning for Code Retrieval and Summarization via Semantic-Preserving Transformations
- Generating Accurate Assert Statements for Unit Test Cases using Pretrained Transformers
- CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning
- Deep Graph Matching and Searching for Semantic Code Retrieval
- CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences
- Automated Conformance Testing for JavaScript Engines via Deep Compiler Fuzzing
- Automated Classification of Overfitting Patches with Statically Extracted Code Features
- In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPT
- Retrieval-Augmented Generation for Code Summarization via Hybrid GNN
- Synchromesh: Reliable code generation from pre-trained language models
- Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection
- No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation
- Leveraging Code Generation to Improve Code Retrieval and Summarization via Dual Learning
- CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X
- MTFuzz: Fuzzing with a Multi-Task Neural Network
- ROSF: Leveraging Information Retrieval and Supervised Learning for Recommending Code Snippets
- Detecting Privacy Requirements from User Stories with NLP Transfer Learning Models
- Unified Pre-training for Program Understanding and Generation
- LeanDojo: Theorem Proving with Retrieval-Augmented Language Models
- CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion Models
- Large Language Models for Software Engineering: Survey and Open Problems
- Detecting Code Clones with Graph Neural Networkand Flow-Augmented Abstract Syntax Tree
- GypSum: Learning Hybrid Representations for Code Summarization
- Montage: A Neural Network Language Model-Guided JavaScript Engine Fuzzer
- Neural-Based Test Oracle Generation: A Large-scale Evaluation and Lessons Learned
- Machine Learning Methods in Solving the Boolean Satisfiability Problem
- Unified Abstract Syntax Tree Representation Learning for Cross-Language Program Classification
- HyperTree Proof Search for Neural Theorem Proving
- ChatUniTest: A Framework for LLM-Based Test Generation
- Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT
- A Survey on Large Language Models for Software Engineering
- Embedding API Dependency Graph for Neural Code Generation
- RMove: Recommending Move Method Refactoring Opportunities using Structural and Semantic Representations of Code
- TranS^3: A Transformer-based Framework for Unifying Code Summarization and Code Search
- Prutor: A System for Tutoring CS1 and Collecting Student Programs for Analysis
- A Study about the Knowledge and Use of Requirements Engineering Standards in Industry
- Using a Nearest-Neighbour, BERT-Based Approach for Scalable Clone Detection
- TreeGen: A Tree-Based Transformer Architecture for Code Generation
- A Grammar-Based Structural CNN Decoder for Code Generation
- Avgust: Automating Usage-Based Test Generation from Videos of App Executions
- Learning Lenient Parsing & Typing via Indirect Supervision
- Styler: learning formatting conventions to repair Checkstyle violations
- Dev2vec: Representing Domain Expertise of Developers in an Embedding Space
- Semantic Parsing with Syntax- and Table-Aware SQL Generation
- Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models
- Practitioners' Expectations on Code Completion
- The Hitchhiker's Guide to Program Analysis: A Journey with Large Language Models
- A Survey on Machine Learning Techniques for Source Code Analysis
- xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval
- CoCoSoDa: Effective Contrastive Learning for Code Search
- Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code
- Multi-task Learning based Pre-trained Language Model for Code Completion
- Unit Test Case Generation with Transformers and Focal Context
- SEED: Semantic Graph based Deep detection for type-4 clone
- On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilot
- Connecting Software Metrics across Versions to Predict Defects
- Cascaded Fast and Slow Models for Efficient Semantic Code Search
- DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs
- Evaluating Representation Learning of Code Changes for Predicting Patch Correctness in Program Repair
- Exploring Dynamic Selection of Branch Expansion Orders for Code Generation
- TransRepair: Context-aware Program Repair for Compilation Errors
- Efficient Mutation Testing via Pre-Trained Language Models
- SkCoder: A Sketch-based Approach for Automatic Code Generation
- Generation-Augmented Query Expansion For Code Retrieval
- Improving Code Summarization with Block-wise Abstract Syntax Tree Splitting
- HAConvGNN: Hierarchical Attention Based Convolutional Graph Neural Network for Code Documentation Generation in Jupyter Notebooks
- Automatic Web Testing using Curiosity-Driven Reinforcement Learning
- Automated Query Reformulation for Efficient Search based on Query Logs From Stack Overflow
- Extracting Concise Bug-Fixing Patches from Human-Written Patches in Version Control Systems
- CoSQA: 20,000+ Web Queries for Code Search and Question Answering
- Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing
- Exploiting Method Names to Improve Code Summarization: A Deliberation Multi-Task Learning Approach
- AI-based Question Answering Assistance for Analyzing Natural-language Requirements
- Modeling Review History for Reviewer Recommendation:A Hypergraph Approach
- Mitigating the Effect of Class Imbalance in Fault Localization Using Context-aware Generative Adversarial Network
- Using Large-scale Heterogeneous Graph Representation Learning for Code Review Recommendations at Microsoft