Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base

Cunxiang Wang, Fuli Luo, Yanyang Li +3

Pre-trained language models (PLMs) like BERT have made significant progress in various downstream NLP tasks. However, by asking models to do cloze-style tests, recent work finds th…