Compositional generalization through meta sequence-to-sequence learning
arXiv:1906.05381
Abstract
People can learn a new concept and use it compositionally, understanding how to "blicket twice" after learning how to "blicket." In contrast, powerful sequence-to-sequence (seq2seq) neural networks fail such tests of compositionality, especially when composing new concepts together with existing concepts. In this paper, I show how memory-augmented neural networks can be trained to generalize compositionally through meta seq2seq learning. In this approach, models train on a series of seq2seq problems to acquire the compositional skills needed to solve new seq2seq problems. Meta se2seq learning solves several of the SCAN tests for compositional learning and can learn to apply implicit rules to variables.
This paper appears in the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada
References in corpus (6)
- Sequence to Sequence Learning with Neural Networks
- Deep Learning: A Critical Appraisal
- Learning to reinforcement learn
- Compositional generalization in a deep seq2seq model by separating syntax and semantics
- Learning to Infer Program Sketches
- Systematic Generalization: What Is Required and Can It Be Learned?
Cited by in corpus (6)
- Environmental drivers of systematicity and generalization in a situated agent
- Grounded Language Learning Fast and Slow
- Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding
- Few-shot Sequence Learning with Transformers
- Complementary Structure-Learning Neural Networks for Relational Reasoning
- Concepts, Properties and an Approach for Compositional Generalization