Open Vocabulary Learning on Source Code with a Graph-Structured Cache

Abstract

Machine learning models that take computer program source code as input typically use Natural Language Processing (NLP) techniques. However, a major challenge is that code is written using an open, rapidly changing vocabulary due to, e.g., the coinage of new variable and method names. Reasoning over such a vocabulary is not something for which most NLP methods are designed. We introduce a Graph-Structured Cache to address this problem; this cache contains a node for each new word the model encounters with edges connecting each word to its occurrences in the code. We find that combining this graph-structured cache strategy with recent Graph-Neural-Network-based models for supervised learning on code improves the models' performance on a code completion task and a variable naming task --- with over $100\%$ relative improvement on the latter --- at the cost of a moderate increase in computation time.

arXiv commentPublished in the International Conference on Machine Learning (ICML 2019), 13 pages

Cited by in corpus (10)

• Code Structure Guided Transformer for Source Code Summarization
• Deep Graph Matching and Searching for Semantic Code Retrieval
• GNN-FiLM: Graph Neural Networks with Feature-wise Linear Modulation
• A Survey on Open Set Recognition
• On the Replicability and Reproducibility of Deep Learning in Software Engineering
• Code Prediction by Feeding Trees to Transformers
• A Survey on Machine Learning Techniques for Source Code Analysis
• Learning to Represent Programs with Heterogeneous Graphs
• Adversarial Training for Code Retrieval with Question-Description Relevance Regularization
• Learning to Extend Program Graphs to Work-in-Progress Code