Precise Learning of Source Code Contextual Semantics via Hierarchical Dependence Structure and Graph Attention Networks

Abstract

Deep learning is being used extensively in a variety of software engineering tasks, e.g., program classification and defect prediction. Although the technique eliminates the required process of feature engineering, the construction of source code model significantly affects the performance on those tasks. Most recent works was mainly focused on complementing AST-based source code models by introducing contextual dependencies extracted from CFG. However, all of them pay little attention to the representation of basic blocks, which are the basis of contextual dependencies. In this paper, we integrated AST and CFG and proposed a novel source code model embedded with hierarchical dependencies. Based on that, we also designed a neural network that depends on the graph attention mechanism.Specifically, we introduced the syntactic structural of the basic block, i.e., its corresponding AST, in source code model to provide sufficient information and fill the gap. We have evaluated this model on three practical software engineering tasks and compared it with other state-of-the-art methods. The results show that our model can significantly improve the performance. For example, compared to the best performing baseline, our model reduces the scale of parameters by 50\% and achieves 4\% improvement on accuracy on program classification task.

arXiv comment17pages, published on Journal of Systems and Software

References in corpus (10)

• Sequence to Sequence Learning with Neural Networks
• Inductive Representation Learning on Large Graphs
• Spectral Networks and Locally Connected Networks on Graphs
• Graph Neural Networks: A Review of Methods and Applications
• Learning Convolutional Neural Networks for Graphs
• Convolutional Neural Networks over Tree Structures for Programming Language Processing
• CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning
• Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach
• Learning to Represent Programs with Graphs
• Learning Blended, Precise Semantic Program Embeddings