Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering
arXiv:2007.00266
Abstract
Answering questions that involve multi-step reasoning requires decomposing them and using the answers of intermediate steps to reach the final answer. However, state-of-the-art models in grounded question answering often do not explicitly perform decomposition, leading to difficulties in generalization to out-of-distribution examples. In this work, we propose a model that computes a representation and denotation for all question spans in a bottom-up, compositional manner using a CKY-style parser. Our model induces latent trees, driven by end-to-end (the answer) supervision only. We show that this inductive bias towards tree structures dramatically improves systematic generalization to out-of-distribution examples, compared to strong baselines on an arithmetic expressions benchmark as well as on CLOSURE, a dataset that focuses on systematic generalization for grounded question answering. On this challenging dataset, our model reaches an accuracy of 96.1%, significantly higher than prior models that almost perfectly solve the task on a random, in-distribution split.
arXiv commentAccepted for publication in Transactions of the Association for Computational Linguistics (TACL), 2020. Author's final version
References in corpus (13)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars
- Layer Normalization
- Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding
- Learning to Execute
- Improving Text-to-SQL Evaluation Methodology
- Neural GPUs Learn Algorithms
- Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs
- Neural Arithmetic Logic Units
- On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law
- A Benchmark for Systematic Generalization in Grounded Language Understanding
- CLOSURE: Assessing Systematic Generalization of CLEVR Models
- Systematic Generalization: What Is Required and Can It Be Learned?