Structured Generative Models of Natural Source Code
arXiv:1401.0514
Abstract
We study the problem of building generative models of natural source code (NSC); that is, source code written and understood by humans. Our primary contribution is to describe a family of generative models for NSC that have three key properties: First, they incorporate both sequential and hierarchical structure. Second, we learn a distributed representation of source code elements. Finally, they integrate closely with a compiler, which allows leveraging compiler logic and abstractions when building structure into the model. We also develop an extension that includes more complex structure, refining how the model generates identifier tokens based on what variables are currently in scope. Our models can be learned efficiently, and we show empirically that including appropriate structure greatly improves the models, measured by the probability of generating test programs.
References in corpus (2)
Cited by in corpus (38)
- Evaluating Large Language Models Trained on Code
- Learning Deep Generative Models of Graphs
- Learning Natural Coding Conventions
- Learning to Execute
- Mining Idioms from Source Code
- A Convolutional Attention Network for Extreme Summarization of Source Code
- CODIT: Code Editing with Tree-Based Neural Models
- A Syntactic Neural Model for General-Purpose Code Generation
- On the Generalizability of Neural Program Models with respect to Semantic-Preserving Program Transformations
- Review Networks for Caption Generation
- Neural Reverse Engineering of Stripped Binaries using Augmented Control Flow Graphs
- Learning Memory Access Patterns
- Generative Code Modeling with Graphs
- Learning to Represent Edits
- Learning Python Code Suggestion with a Sparse Pointer Network
- StructureNet: Hierarchical Graph Networks for 3D Shape Generation
- Abstract Syntax Networks for Code Generation and Semantic Parsing
- CodeGRU: Context-aware Deep Learning with Gated Recurrent Unit for Source Code Modeling
- Learning to Generate Corrective Patches using Neural Machine Translation
- Graphite: Iterative Generative Modeling of Graphs
- Learning API Usages from Bytecode: A Statistical Approach
- Neural Attribute Machines for Program Generation
- SmartPaste: Learning to Adapt Source Code
- Neural Networks for Modeling Source Code Edits
- Deep Learning to Detect Redundant Method Comments
- Evaluation of Generalizability of Neural Program Analyzers under Semantic-Preserving Transformations
- Towards Neural Decompilation
- Learning a Static Bug Finder from Data
- Learning Probabilistic Programs
- A Generative Model of Software Dependency Graphs to Better Understand Software Evolution
- Tailored Mutants Fit Bugs Better
- TreeGAN: Syntax-Aware Sequence Generation with Generative Adversarial Networks
- Program Classification Using Gated Graph Attention Neural Network for Online Programming Service
- Recommendation of Exception Handling Code in Mobile App Development
- On-the-Fly Adaptation of Source Code Models using Meta-Learning
- Abstraction Refinement Guided by a Learnt Probabilistic Model
- Improving Automatic Source Code Summarization via Deep Reinforcement Learning
- Neural Program Generation Modulo Static Analysis