Rapidly Bootstrapping a Question Answering Dataset for COVID-19

Abstract

We present CovidQA, the beginnings of a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered from Kaggle's COVID-19 Open Research Dataset Challenge. To our knowledge, this is the first publicly available resource of its type, and intended as a stopgap measure for guiding research until more substantial evaluation resources become available. While this dataset, comprising 124 question-article pairs as of the present version 0.1 release, does not have sufficient examples for supervised machine learning, we believe that it can be helpful for evaluating the zero-shot or transfer capabilities of existing models on topics specifically related to COVID-19. This paper describes our methodology for constructing the dataset and presents the effectiveness of a number of baselines, including term-based techniques and various transformer-based models. The dataset is available at http://covidqa.ai/

References in corpus (2)

• Cascade Ranking for Operational E-commerce Search
• Rapidly Deploying a Neural Search Engine for the COVID-19 Open Research Dataset: Preliminary Thoughts and Lessons Learned

Cited by in corpus (5)

• CO-Search: COVID-19 Information Retrieval with Semantic Search, Question Answering, and Abstractive Summarization
• Coronavirus Knowledge Graph: A Case Study
• orgFAQ: A New Dataset and Analysis on Organizational FAQs and User Questions
• Repurposing TREC-COVID Annotations to Answer the Key Questions of CORD-19
• Can questions summarize a corpus? Using question generation for characterizing COVID-19 research