Learning to Represent Programs with Graphs

Abstract

Learning tasks on source code (i.e., formal languages) have been considered recently, but most work has tried to transfer natural language methods and does not capitalize on the unique opportunities offered by code's known syntax. For example, long-range dependencies induced by using the same variable or function in distant locations are often not considered. We propose to use graphs to represent both the syntactic and semantic structure of code and use graph-based deep learning methods to learn to reason over program structures. In this work, we present how to construct graphs from source code and how to scale Gated Graph Neural Networks training to such large graphs. We evaluate our method on two tasks: VarNaming, in which a network attempts to predict the name of a variable given its usage, and VarMisuse, in which the network learns to reason about selecting the correct variable that should be used at a given program location. Our comparison to methods that use less structured program representations shows the advantages of modeling known structure, and suggests that our models learn to infer meaningful names and to solve the VarMisuse task in many cases. Additionally, our testing showed that VarMisuse identifies a number of bugs in mature open-source projects.

Published in ICLR 2018. arXiv admin note: text overlap with arXiv:1705.07867

Cited by in corpus (11)

• On the Feasibility of Transfer-learning Code Smells using Deep Learning
• LambdaNet: Probabilistic Type Inference using Graph Neural Networks
• On the Replicability and Reproducibility of Deep Learning in Software Engineering
• Precise Learning of Source Code Contextual Semantics via Hierarchical Dependence Structure and Graph Attention Networks
• Embedding API Dependency Graph for Neural Code Generation
• The Remarkable Role of Similarity in Redundancy-based Program Repair
• DeepDebug: Fixing Python Bugs Using Stack Traces, Backtranslation, and Code Skeletons
• OptTyper: Probabilistic Type Inference by Optimising Logical and Natural Constraints
• Assessing the Effectiveness of Syntactic Structure to Learn Code Edit Representations
• Modeling Programs Hierarchically with Stack-Augmented LSTM
• Learning, transferring, and recommending performance knowledge with Monte Carlo tree search and neural networks