A deep language model for software code

Abstract

Existing language models such as n-grams for software code often fail to capture a long context where dependent code elements scatter far apart. In this paper, we propose a novel approach to build a language model for software code to address this particular issue. Our language model, partly inspired by human memory, is built upon the powerful deep learning-based Long Short Term Memory architecture that is capable of learning long-term dependencies which occur frequently in software code. Results from our intrinsic evaluation on a corpus of Java projects have demonstrated the effectiveness of our language model. This work contributes to realizing our vision for DeepSoft, an end-to-end, generic deep learning-based framework for modeling software and its development process.

References in corpus (2)

• Convolutional Neural Networks over Tree Structures for Programming Language Processing
• DeepSoft: A vision for a deep model of software

Cited by in corpus (8)

• Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code
• VUDENC: Vulnerability Detection with Deep Learning on a Natural Codebase for Python
• Automatic feature learning for vulnerability prediction
• Maybe Deep Neural Networks are the Best Choice for Modeling Source Code
• OptTyper: Probabilistic Type Inference by Optimising Logical and Natural Constraints
• Language Modelling for Source Code with Transformer-XL
• A Comparison of Code Embeddings and Beyond
• On-the-Fly Adaptation of Source Code Models using Meta-Learning