Learning to Extend Program Graphs to Work-in-Progress Code
arXiv:2105.14038
Abstract
Source code spends most of its time in a broken or incomplete state during software development. This presents a challenge to machine learning for code, since high-performing models typically rely on graph structured representations of programs derived from traditional program analyses. Such analyses may be undefined for broken or incomplete code. We extend the notion of program graphs to work-in-progress code by learning to predict edge relations between tokens, training on well-formed code before transferring to work-in-progress code. We consider the tasks of code completion and localizing and repairing variable misuse in a work-in-process scenario. We demonstrate that training relation-aware models with fine-tuned edges consistently leads to improved performance on both tasks.
References in corpus (23)
- Adam: A Method for Stochastic Optimization
- Categorical Reparameterization with Gumbel-Softmax
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Convolutional Neural Networks over Tree Structures for Programming Language Processing
- Neural Relational Inference for Interacting Systems
- Learning and Evaluating Contextual Embedding of Source Code
- Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code
- A Syntactic Neural Model for General-Purpose Code Generation
- Generative Code Modeling with Graphs
- ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing
- Graph-based, Self-Supervised Program Repair from Diagnostic Feedback
- Maybe Deep Neural Networks are the Best Choice for Modeling Source Code
- Structural Language Models of Code
- Abstract Syntax Networks for Code Generation and Semantic Parsing
- Code Prediction by Feeding Trees to Transformers
- ProGraML: Graph-based Deep Learning for Program Optimization and Analysis
- Gradient Estimation with Stochastic Softmax Tricks
- Open Vocabulary Learning on Source Code with a Graph-Structured Cache
- Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networks
- Towards Full-line Code Completion with Neural Language Models
- Learning Graph Structure With A Finite-State Automaton Layer