WikiCheck: An end-to-end open source Automatic Fact-Checking API based on Wikipedia
arXiv:2109.00835
Abstract
With the growth of fake news and disinformation, the NLP community has been working to assist humans in fact-checking. However, most academic research has focused on model accuracy without paying attention to resource efficiency, which is crucial in real-life scenarios. In this work, we review the State-of-the-Art datasets and solutions for Automatic Fact-checking and test their applicability in production environments. We discover overfitting issues in those models, and we propose a data filtering method that improves the model's performance and generalization. Then, we design an unsupervised fine-tuning of the Masked Language models to improve its accuracy working with Wikipedia. We also propose a novel query enhancing method to improve evidence discovery using the Wikipedia Search API. Finally, we present a new fact-checking system, the \textit{WikiCheck} API that automatically performs a facts validation process based on the Wikipedia knowledge base. It is comparable to SOTA solutions in terms of accuracy and can be used on low-memory CPU instances.
References in corpus (12)
- Universal Sentence Encoder
- Enhanced LSTM for Natural Language Inference
- ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
- Pointer Sentinel Mixture Models
- Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
- Multi-Task Deep Neural Networks for Natural Language Understanding
- Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems
- Combining Fact Extraction and Verification with Neural Semantic Matching Networks
- Learning by Semantic Similarity Makes Abstractive Summarization Better
- Online Disinformation and the Role of Wikipedia
- WhatTheWikiFact: Fact-Checking Claims Against Wikipedia