A Literature Study of Embeddings on Source Code

Abstract

Natural language processing has improved tremendously after the success of word embedding techniques such as word2vec. Recently, the same idea has been applied on source code with encouraging results. In this survey, we aim to collect and discuss the usage of word embedding techniques on programs and source code. The articles in this survey have been collected by asking authors of related work and with an extensive search on Google Scholar. Each article is categorized into five categories: 1. embedding of tokens 2. embedding of functions or methods 3. embedding of sequences or sets of method calls 4. embedding of binary code 5. other embeddings. We also provide links to experimental data and show some remarkable visualization of code embeddings. In summary, word embedding has been successfully applied on different granularities of source code. With access to countless open-source repositories, we see a great potential of applying other data-driven natural language processing techniques on source code in the future.

References in corpus (9)

• Sorting and Transforming Program Repair Ingredients via Deep Learning Code Similarities
• Automated software vulnerability detection with machine learning
• Learning to Represent Edits
• Dynamic Neural Program Embedding for Program Repair
• Learning Continuous Semantic Representations of Symbolic Expressions
• Semantic Code Repair using Neuro-Symbolic Transformation Networks
• The Remarkable Role of Similarity in Redundancy-based Program Repair
• Path-Based Function Embedding and its Application to Specification Mining
• A Cross-Architecture Instruction Embedding Model for Natural Language Processing-Inspired Binary Code Analysis

Cited by in corpus (14)

• Neural Program Repair with Execution-based Backpropagation
• Commit2Vec: Learning Distributed Representations of Code Changes
• SCELMo: Source Code Embeddings from Language Models
• ProGraML: Graph-based Deep Learning for Program Optimization and Analysis
• Towards Demystifying Dimensions of Source Code Embeddings
• InferCode: Self-Supervised Learning of Code Representations by Predicting Subtrees
• TransformCode: A Contrastive Learning Framework for Code Embedding via Subtree Transformation
• Code2Snapshot: Using Code Snapshots for Learning Representations of Source Code
• Combining Code Embedding with Static Analysis for Function-Call Completion
• A Comparison of Code Embeddings and Beyond
• Rethinking complexity for software code structures: A pioneering study on Linux kernel code repository
• Universal Policies for Software-Defined MDPs
• Adversarial Training for Code Retrieval with Question-Description Relevance Regularization
• An Evalutation of Programming Language Models' performance on Software Defect Detection