RACE: Large-scale ReAding Comprehension Dataset From Examinations
arXiv:1704.04683
Abstract
We present RACE, a new dataset for benchmark evaluation of methods in the reading comprehension task. Collected from the English exams for middle and high school Chinese students in the age range between 12 to 18, RACE consists of near 28,000 passages and near 100,000 questions generated by human experts (English instructors), and covers a variety of topics which are carefully designed for evaluating the students' ability in understanding and reasoning. In particular, the proportion of questions that requires reasoning is much larger in RACE than that in other benchmark datasets for reading comprehension, and there is a significant gap between the performance of the state-of-the-art models (43%) and the ceiling human performance (95%). We hope this new dataset can serve as a valuable resource for research and evaluation in machine comprehension. The dataset is freely available at http://www.cs.cmu.edu/~glai1/data/race/ and the code is available at https://github.com/qizhex/RACE_AR_baselines.
EMNLP 2017
References in corpus (3)
Cited by in corpus (68)
- Language Models are Few-Shot Learners
- XLNet: Generalized Autoregressive Pretraining for Language Understanding
- Multitask Prompted Training Enables Zero-Shot Task Generalization
- BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
- Transforming Question Answering Datasets Into Natural Language Inference Datasets
- Arabic Offensive Language on Twitter: Analysis and Experiments
- Multi-hop Question Answering via Reasoning Chains
- Memory-Efficient Pipeline-Parallel DNN Training
- A Survey on Transfer Learning in Natural Language Processing
- DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
- Explaining Question Answering Models through Text Generation
- Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions
- Machine Reading Comprehension: a Literature Review
- ETC: Encoding Long and Structured Inputs in Transformers
- Efficient and Robust Question Answering from Minimal Context over Documents
- A Co-Matching Model for Multi-choice Reading Comprehension
- SG-Net: Syntax-Guided Machine Reading Comprehension
- Learning to Encode Position for Transformer with Continuous Dynamical Model
- Optimal Subarchitecture Extraction For BERT
- Dialog State Tracking: A Neural Reading Comprehension Approach
- Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering
- Dynamic Fusion Networks for Machine Reading Comprehension
- Difficulty Controllable Generation of Reading Comprehension Questions
- Review Conversational Reading Comprehension
- Multi-branch Attentive Transformer
- What Makes Reading Comprehension Questions Easier?
- Generating Distractors for Reading Comprehension Questions from Real Examinations
- Improving Question Answering by Commonsense-Based Pre-Training
- MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension
- Multilingual Question Answering from Formatted Text applied to Conversational Agents
- Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension
- ExpMRC: Explainability Evaluation for Machine Reading Comprehension
- Combining pre-trained language models and structured knowledge
- Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based models
- On the comparability of Pre-trained Language Models
- Incorporating Structured Commonsense Knowledge in Story Completion
- DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering
- Controlling Risk of Web Question Answering
- Reasoning Over Paragraph Effects in Situations
- Cross-Lingual Transfer Learning for Question Answering
- Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning
- Counterfactual Variable Control for Robust and Interpretable Question Answering
- Multi-class Text Classification using BERT-based Active Learning
- ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion
- Reading Comprehension as Natural Language Inference: A Semantic Analysis
- Testing pre-trained Transformer models for Lithuanian news clustering
- SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval
- Large-scale Cloze Test Dataset Created by Teachers
- Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Task
- PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolution
- On Extending NLP Techniques from the Categorical to the Latent Space: KL Divergence, Zipf's Law, and Similarity Search
- Learning to Ask Screening Questions for Job Postings
- GenNet : Reading Comprehension with Multiple Choice Questions using Generation and Selection model
- Answering questions by learning to rank -- Learning to rank by answering questions
- Controllable Open-ended Question Generation with A New Question Type Ontology
- Co-Attention Hierarchical Network: Generating Coherent Long Distractors for Reading Comprehension
- ManyModalQA: Modality Disambiguation and QA over Diverse Inputs
- Situation and Behavior Understanding by Trope Detection on Films
- FAT ALBERT: Finding Answers in Large Texts using Semantic Similarity Attention Layer based on BERT
- Zero-Shot Open-Book Question Answering
- Deep Human Answer Understanding for Natural Reverse QA
- NEUer at SemEval-2021 Task 4: Complete Summary Representation by Filling Answers into Question for Matching Reading Comprehension
- Challenge Closed-book Science Exam: A Meta-learning Based Question Answering System
- A BERT-based Distractor Generation Scheme with Multi-tasking and Negative Answer Training Strategies
- SCDE: Sentence Cloze Dataset with High Quality Distractors From Examinations
- Zero-shot Task Transfer for Invoice Extraction via Class-aware QA Ensemble
- ODSQA: Open-domain Spoken Question Answering Dataset
- An Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering