Code Completion with Neural Attention and Pointer Networks

Abstract

Intelligent code completion has become an essential research task to accelerate modern software development. To facilitate effective code completion for dynamically-typed programming languages, we apply neural language models by learning from large codebases, and develop a tailored attention mechanism for code completion. However, standard neural language models even with attention mechanism cannot correctly predict the out-of-vocabulary (OoV) words that restrict the code completion performance. In this paper, inspired by the prevalence of locally repeated terms in program source code, and the recently proposed pointer copy mechanism, we propose a pointer mixture network for better predicting OoV words in code completion. Based on the context, the pointer mixture network learns to either generate a within-vocabulary word through an RNN component, or regenerate an OoV word from local context through a pointer component. Experiments on two benchmarked datasets demonstrate the effectiveness of our attention mechanism and pointer mixture network on the code completion task.

Accepted in IJCAI 2018

References in corpus (1)

• A Convolutional Attention Network for Extreme Summarization of Source Code

Cited by in corpus (7)

• Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code
• An Empirical Study on the Usage of Transformer Models for Code Completion
• Empirical Study of Transformers for Source Code
• Senatus -- A Fast and Accurate Code-to-Code Recommendation Engine
• On the Applicability of Language Models to Block-Based Programs
• A Closer Look into Transformer-Based Code Intelligence Through Code Transformation: Challenges and Opportunities
• Data-Driven Approach for Log Instruction Quality Assessment