Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks

Abstract

Systematic compositionality is the ability to recombine meaningful units with regular and predictable outcomes, and it's seen as key to humans' capacity for generalization in language. Recent work has studied systematic compositionality in modern seq2seq models using generalization to novel navigation instructions in a grounded environment as a probing tool, requiring models to quickly bootstrap the meaning of new words. We extend this framework here to settings where the model needs only to recombine well-trained functional words (such as "around" and "right") in novel contexts. Our findings confirm and strengthen the earlier ones: seq2seq models can be impressively good at generalizing to novel combinations of previously-seen input, but only when they receive extensive training on the specific pattern to be generalized (e.g., generalizing from many examples of "X around right" to "jump around right"), while failing when generalization requires novel application of compositional rules (e.g., inferring the meaning of "around right" from those of "right" and "around").

References in corpus (3)

• Sequence to Sequence Learning with Neural Networks
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• One-shot and few-shot learning of word embeddings

Cited by in corpus (10)

• Compositional generalization in a deep seq2seq model by separating syntax and semantics
• Discrete and continuous representations and processing in deep learning: Looking forward
• Human few-shot learning of compositional instructions
• Compositionality decomposed: how do neural networks generalise?
• What they do when in doubt: a study of inductive biases in seq2seq learners
• Structured Reordering for Modeling Latent Alignments in Sequence Transduction
• Compositional Generalization for Primitive Substitutions
• Compositional Generalization via Neural-Symbolic Stack Machines
• Analogical Reasoning for Visually Grounded Language Acquisition
• The relational processing limits of classic and contemporary neural network models of language processing