Compositional Generalization for Primitive Substitutions

Abstract

Compositional generalization is a basic mechanism in human language learning, but current neural networks lack such ability. In this paper, we conduct fundamental research for encoding compositionality in neural networks. Conventional methods use a single representation for the input sentence, making it hard to apply prior knowledge of compositionality. In contrast, our approach leverages such knowledge with two representations, one generating attention maps, and the other mapping attended input words to output symbols. We reduce the entropy in each representation to improve generalization. Our experiments demonstrate significant improvements over the conventional methods in five NLP tasks including instruction learning and machine translation. In the SCAN domain, it boosts accuracies from 14.0% to 98.8% in Jump task, and from 92.0% to 99.7% in TurnLeft task. It also beats human performance on a few-shot learning task. We hope the proposed approach can help ease future research towards human-level compositional language learning.

EMNLP 2019

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• TensorFlow: A system for large-scale machine learning
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• VQA: Visual Question Answering
• Continual Learning Through Synaptic Intelligence
• Efficient Lifelong Learning with A-GEM
• Memorize or generalize? Searching for a compositional RNN in a haystack
• Human few-shot learning of compositional instructions
• Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks

Cited by in corpus (6)

• Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures
• Analogical Reasoning for Visually Grounded Language Acquisition
• Sequence-to-Sequence Networks Learn the Meaning of Reflexive Anaphora
• Compositional Attention: Disentangling Search and Retrieval
• Concepts, Properties and an Approach for Compositional Generalization
• Learning Algebraic Recombination for Compositional Generalization