Understanding Long Programming Languages with Structure-Aware Sparse Attention

Tingting Liu, Chengyu Wang, Cen Chen +2

Programming-based Pre-trained Language Models (PPLMs) such as CodeBERT have achieved great success in many downstream code-related tasks. Since the memory and computational complex…