A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets

Abstract

Low-quality data can cause downstream problems in high-stakes applications. Data-centric approach emphasizes on improving dataset quality to enhance model performance. High-quality datasets are needed for general-purpose Large Language Models (LLMs) training, as well as for domain-specific models, which are usually small in size as it is costly to engage a large number of domain experts for their creation. Thus, it is vital to ensure high-quality domain-specific training data. In this paper, we propose a framework for enhancing the data quality of original datasets. We applied the proposed framework to four biomedical datasets and showed relative improvement of up to 33%/40% for fine-tuning of retrieval/reader models on the BioASQ dataset when using back translation to enhance the original dataset quality.

References in corpus (9)

• BERTScore: Evaluating Text Generation with BERT
• Learning the Difference that Makes a Difference with Counterfactually-Augmented Data
• Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
• Teaching language models to support answers with verified quotes
• Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• DataCLUE: A Benchmark Suite for Data-centric NLP
• Fix your Models by Fixing your Datasets
• Query-focused Extractive Summarisation for Biomedical and COVID-19 Complex Question Answering