77 citations · 245 across the 16 of their papers we have counts for
3 papers · 2 filters
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model
Wenhan Xiong, Jingfei Du, William Yang Wang +1
Recent breakthroughs of pretrained language models have shown the effectiveness of self-supervised learning for a wide range of natural language processing (NLP) tasks. In addition…
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Yinhan Liu, Myle Ott, Naman Goyal +7
Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging. Training is computationally expensive, often…
Knowledge-Augmented Language Model and its Application to Unsupervised Named-Entity Recognition
Angli Liu, Jingfei Du, Veselin Stoyanov
Traditional language models are unable to efficiently model entity names observed in text. All but the most popular named entities appear infrequently in text providing insufficien…