SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoning
arXiv:2104.05832
Abstract
This paper proposes a question-answering (QA) benchmark for spatial reasoning on natural language text which contains more realistic spatial phenomena not covered by prior work and is challenging for state-of-the-art language models (LM). We propose a distant supervision method to improve on this task. Specifically, we design grammar and reasoning rules to automatically generate a spatial description of visual scenes and corresponding QA pairs. Experiments show that further pretraining LMs on these automatically generated data significantly improves LMs' capability on spatial understanding, which in turn helps to better solve two external datasets, bAbI, and boolQ. We hope that this work can foster investigations into more sophisticated models for spatial reasoning over text.
arXiv commentNAACL 2021
References in corpus (5)
- Decoupled Weight Decay Regularization
- Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks
- What do Models Learn from Question Answering Datasets?
- GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Level
- From Spatial Relations to Spatial Configurations