CodeArt: Better Code Models by Attention Regularization When Symbols Are Lacking

Zian Su, Xiangzhe Xu, Ziyang Huang +4

Transformer based code models have impressive performance in many software engineering tasks. However, their effectiveness degrades when symbols are missing or not informative. The…