GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Level
arXiv:1908.07855
Abstract
Scenario-based question answering (SQA) has attracted increasing research attention. It typically requires retrieving and integrating knowledge from multiple sources, and applying general knowledge to a specific case described by a scenario. SQA widely exists in the medical, geography, and legal domains---both in practice and in the exams. In this paper, we introduce the GeoSQA dataset. It consists of 1,981 scenarios and 4,110 multiple-choice questions in the geography domain at high school level, where diagrams (e.g., maps, charts) have been manually annotated with natural language descriptions to benefit NLP research. Benchmark results on a variety of state-of-the-art methods for question answering, textual entailment, and reading comprehension demonstrate the unique challenges presented by SQA for future research.
arXiv comment6 pages, to appear at the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019)
References in corpus (5)
- Enhanced LSTM for Natural Language Inference
- Natural Language Inference over Interaction Space
- CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction
- XNLI: Evaluating Cross-lingual Sentence Representations
- Interpretable Charge Predictions for Criminal Cases: Learning to Generate Court Views from Fact Descriptions