Rapidly Bootstrapping a Question Answering Dataset for COVID-19

Abstract

We present CovidQA, the beginnings of a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered from Kaggle's COVID-19 Open Research Dataset Challenge. To our knowledge, this is the first publicly available resource of its type, and intended as a stopgap measure for guiding research until more substantial evaluation resources become available. While this dataset, comprising 124 question-article pairs as of the present version 0.1 release, does not have sufficient examples for supervised machine learning, we believe that it can be helpful for evaluating the zero-shot or transfer capabilities of existing models on topics specifically related to COVID-19. This paper describes our methodology for constructing the dataset and presents the effectiveness of a number of baselines, including term-based techniques and various transformer-based models. The dataset is available at http://covidqa.ai/

References in corpus (4)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Cascade Ranking for Operational E-commerce Search
• Rapidly Deploying a Neural Search Engine for the COVID-19 Open Research Dataset: Preliminary Thoughts and Lessons Learned

Cited by in corpus (20)

• Biomedical Question Answering: A Survey of Approaches and Challenges
• Study of Different Deep Learning Approach with Explainable AI for Screening Patients with COVID-19 Symptoms: Using CT Scan and Chest X-ray Image Dataset
• Pre-trained Language Models in Biomedical Domain: A Systematic Survey
• CO-Search: COVID-19 Information Retrieval with Semantic Search, Question Answering, and Abstractive Summarization
• Transformer-Based Models for Question Answering on COVID19
• End-to-End QA on COVID-19: Domain Adaptation with Synthetic Training
• Coronavirus Knowledge Graph: A Case Study
• CAiRE-COVID: A Question Answering and Query-focused Multi-Document Summarization System for COVID-19 Scholarly Information Management
• Meta-research on COVID-19: An overview of the early trends
• Hidden Backdoors in Human-Centric Language Models
• Scientific Claim Verification with VERT5ERINI
• Contextual embedding and model weighting by fusing domain knowledge on Biomedical Question Answering
• A Comparative Study on Transfer Learning and Distance Metrics in Semantic Clustering over the COVID-19 Tweets
• Repurposing TREC-COVID Annotations to Answer the Key Questions of CORD-19
• orgFAQ: A New Dataset and Analysis on Organizational FAQs and User Questions
• Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies
• COVIDRead: A Large-scale Question Answering Dataset on COVID-19
• Open-Domain Question-Answering for COVID-19 and Other Emergent Domains
• UCD-CS at W-NUT 2020 Shared Task-3: A Text to Text Approach for COVID-19 Event Extraction on Social Media
• Can questions summarize a corpus? Using question generation for characterizing COVID-19 research