Reddit Entity Linking Dataset

Abstract

We introduce and make publicly available an entity linking dataset from Reddit that contains 17,316 linked entities, each annotated by three human annotators and then grouped into Gold, Silver, and Bronze to indicate inter-annotator agreement. We analyze the different errors and disagreements made by annotators and suggest three types of corrections to the raw data. Finally, we tested existing entity linking models that are trained and tuned on text from non-social media datasets. We find that, although these existing entity linking models perform very well on their original datasets, they perform poorly on this social media dataset. We also show that the majority of these errors can be attributed to poor performance on the mention detection subtask. These results indicate the need for better entity linking models that can be applied to the enormous amount of social media text.

20 pages and 4 figures

References in corpus (10)

• A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
• Analysis of Named Entity Recognition and Linking for Tweets
• Neural Entity Linking: A Survey of Models Based on Deep Learning
• A Large Self-Annotated Corpus for Sarcasm
• Context-Dependent Fine-Grained Entity Type Tagging
• Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking
• DeepType: Multilingual Entity Linking by Neural Type System Evolution
• REL: An Entity Linker Standing on the Shoulders of Giants
• Twitter for Sparking a Movement, Reddit for Sharing the Moment: #metoo through the Lens of Social Media
• Neural Architectures for Fine-grained Entity Type Classification