DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications

Abstract

This paper introduces DuReader, a new large-scale, open-domain Chinese ma- chine reading comprehension (MRC) dataset, designed to address real-world MRC. DuReader has three advantages over previous MRC datasets: (1) data sources: questions and documents are based on Baidu Search and Baidu Zhidao; answers are manually generated. (2) question types: it provides rich annotations for more question types, especially yes-no and opinion questions, that leaves more opportunity for the research community. (3) scale: it contains 200K questions, 420K answers and 1M documents; it is the largest Chinese MRC dataset so far. Experiments show that human performance is well above current state-of-the-art baseline systems, leaving plenty of room for the community to make improvements. To help the community make these improvements, both DuReader and baseline systems have been posted online. We also organize a shared competition to encourage the exploration of more models. Since the release of the task, there are significant improvements over the baselines.

arXiv comment10 pages, ACL 2018 MRQA Workshop camera-ready version

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Teaching Machines to Read and Comprehend
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
• Dynamic Coattention Networks For Question Answering
• Machine Comprehension Using Match-LSTM and Answer Pointer
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• Adversarial Examples for Evaluating Reading Comprehension Systems
• A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
• RACE: Large-scale ReAding Comprehension Dataset From Examinations
• Consensus Attention-based Neural Networks for Chinese Reading Comprehension

Cited by in corpus (20)

• ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
• PanGu-$\alpha$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
• DRCD: a Chinese Machine Reading Comprehension Dataset
• ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
• Multilingual Extractive Reading Comprehension by Runtime Machine Translation
• TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages
• Pirá: A Bilingual Portuguese-English Dataset for Question-Answering about the Ocean
• Automatic Judgment Prediction via Legal Reading Comprehension
• Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification
• ERNIE-Doc: A Retrospective Long-Document Modeling Transformer
• Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension
• Knowledge-Aided Open-Domain Question Answering
• ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding
• Cross-Lingual Transfer Learning for Question Answering
• ReviewQA: a relational aspect-based opinion reading dataset
• Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Task
• Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation
• Are Multilingual BERT models robust? A Case Study on Adversarial Attacks for Multilingual Question Answering
• A Study of the Tasks and Models in Machine Reading Comprehension
• Hierarchical Attention: What Really Counts in Various NLP Tasks