Enhancing Source Code Classification Effectiveness via Prompt Learning Incorporating Knowledge Features

Abstract

Researchers have investigated the potential of leveraging pre-trained language models, such as CodeBERT, to enhance source code-related tasks. Previous methodologies have relied on CodeBERT's '[CLS]' token as the embedding representation of input sequences for task performance, necessitating additional neural network layers to enhance feature representation, which in turn increases computational expenses. These approaches have also failed to fully leverage the comprehensive knowledge inherent within the source code and its associated text, potentially limiting classification efficacy. We propose CodeClassPrompt, a text classification technique that harnesses prompt learning to extract rich knowledge associated with input sequences from pre-trained models, thereby eliminating the need for additional layers and lowering computational costs. By applying an attention mechanism, we synthesize multi-layered knowledge into task-specific features, enhancing classification accuracy. Our comprehensive experimentation across four distinct source code-related tasks reveals that CodeClassPrompt achieves competitive performance while significantly reducing computational overhead.

Accepted by Scientific Reports

References in corpus (7)

• XGBoost: A Scalable Tree Boosting System
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Pre-trained Models for Natural Language Processing: A Survey
• Automated software vulnerability detection with machine learning
• On the Feasibility of Transfer-learning Code Smells using Deep Learning
• How to Identify Class Comment Types? A Multi-language Approach for Class Comment Classification
• Analyzing Code Comments to Boost Program Comprehension