Hierarchical Poset Decoding for Compositional Generalization in Language
arXiv:2010.07792
Abstract
We formalize human language understanding as a structured prediction task where the output is a partially ordered set (poset). Current encoder-decoder architectures do not take the poset structure of semantics into account properly, thus suffering from poor compositional generalization ability. In this paper, we propose a novel hierarchical poset decoding paradigm for compositional generalization in language. Intuitively: (1) the proposed paradigm enforces partial permutation invariance in semantics, thus avoiding overfitting to bias ordering information; (2) the hierarchical mechanism allows to capture high-level structures of posets. We evaluate our proposed decoder on Compositional Freebase Questions (CFQ), a large and realistic natural language question answering dataset that is specifically designed to measure compositional generalization. Results show that it outperforms current decoders.
arXiv commentAccepted by Neurips 2020
References in corpus (11)
- Sequence to Sequence Learning with Neural Networks
- Enhanced LSTM for Natural Language Inference
- Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
- Learning Representations and Generative Models for 3D Point Clouds
- SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning
- Compositional generalization in a deep seq2seq model by separating syntax and semantics
- Deep Set Prediction Networks
- Measuring Compositional Generalization: A Comprehensive Method on Realistic Data
- Deep Perm-Set Net: Learn to predict sets with unknown permutation and cardinality using deep neural networks
- Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization
- Neural Program Search: Solving Programming Tasks from Description and Examples