Code Structure Guided Transformer for Source Code Summarization

Abstract

Code summaries help developers comprehend programs and reduce their time to infer the program functionalities during software maintenance. Recent efforts resort to deep learning techniques such as sequence-to-sequence models for generating accurate code summaries, among which Transformer-based approaches have achieved promising performance. However, effectively integrating the code structure information into the Transformer is under-explored in this task domain. In this paper, we propose a novel approach named SG-Trans to incorporate code structural properties into Transformer. Specifically, we inject the local symbolic information (e.g., code tokens and statements) and global syntactic structure (e.g., data flow graph) into the self-attention module of Transformer as inductive bias. To further capture the hierarchical characteristics of code, the local information and global structure are designed to distribute in the attention heads of lower layers and high layers of Transformer. Extensive evaluation shows the superior performance of SG-Trans over the state-of-the-art approaches. Compared with the best-performing baseline, SG-Trans still improves 1.4% and 2.0% in terms of METEOR score, a metric widely used for measuring generation quality, respectively on two benchmark datasets.

References in corpus (10)

• GraphCodeBERT: Pre-training Code Representations with Data Flow
• Structured Neural Summarization
• Contrastive Code Representation Learning
• A parallel corpus of Python functions and documentation strings for automated code documentation and code generation
• CRaDLe: Deep Code Retrieval Based on Semantic Dependency Learning
• Tree-to-Sequence Attentional Neural Machine Translation
• Open Vocabulary Learning on Source Code with a Graph-Structured Cache
• Automatic Generation of Text Descriptive Comments for Code Blocks
• Multi-style Generative Reading Comprehension
• Automatic Source Code Summarization with Extended Tree-LSTM

Cited by in corpus (10)

• What Makes Good In-context Demonstrations for Code Intelligence Tasks with LLMs?
• EyeTrans: Merging Human and Machine Attention for Neural Code Summarization
• Learning in the Wild: Towards Leveraging Unlabeled Data for Effectively Tuning Pre-trained Code Models
• ProTo: Program-Guided Transformer for Program-Guided Tasks
• What Makes a Good TODO Comment?
• Can Developers Prompt? A Controlled Experiment for Code Documentation Generation
• TransformCode: A Contrastive Learning Framework for Code Embedding via Subtree Transformation
• A Closer Look into Transformer-Based Code Intelligence Through Code Transformation: Challenges and Opportunities
• Variational Prefix Tuning for Diverse and Accurate Code Summarization Using Pre-trained Language Models
• Automated Description Generation for Software Patches