Improving BERT with Syntax-aware Local Attention

Zhongli Li, Qingyu Zhou, Chao Li +2

Pre-trained Transformer-based neural language models, such as BERT, have achieved remarkable results on varieties of NLP tasks. Recent works have shown that attention-based models…