An Exploratory Study on Code Attention in BERT

Rishab Sharma, Fuxiang Chen, Fatemeh Fard +1

Many recent models in software engineering introduced deep neural models based on the Transformer architecture or use transformer-based Pre-trained Language Models (PLM) trained on…