Compositional Generalization by Learning Analytical Expressions

Abstract

Compositional generalization is a basic and essential intellective capability of human beings, which allows us to recombine known parts readily. However, existing neural network based models have been proven to be extremely deficient in such a capability. Inspired by work in cognition which argues compositionality can be captured by variable slots with symbolic functions, we present a refreshing view that connects a memory-augmented neural model with analytical expressions, to achieve compositional generalization. Our model consists of two cooperative neural modules, Composer and Solver, fitting well with the cognitive argument while being able to be trained in an end-to-end manner via a hierarchical reinforcement learning algorithm. Experiments on the well-known benchmark SCAN demonstrate that our model seizes a great ability of compositional generalization, solving all challenges addressed by previous works with 100% accuracies.

arXiv commentTo appear in NeurIPS 2020 (Spotlight)

References in corpus (16)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks
• ADADELTA: An Adaptive Learning Rate Method
• End-To-End Memory Networks
• Universal Transformers
• The Optimal Reward Baseline for Gradient-Based Reinforcement Learning
• Curriculum Loss: Robust Learning and Generalization against Label Corruption
• Language as an Abstraction for Hierarchical Deep Reinforcement Learning
• Compositional generalization in a deep seq2seq model by separating syntax and semantics
• Measuring Compositional Generalization: A Comprehensive Method on Realistic Data
• Tree-structured composition in neural networks without tree-structured architectures
• Towards Synthesizing Complex Programs from Input-Output Examples
• Learning Compositional Rules via Neural Program Synthesis
• Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization
• A Split-and-Recombine Approach for Follow-up Query Analysis

Cited by in corpus (8)

• Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures
• Learning Symbolic Rules for Reasoning in Quasi-Natural Language
• The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers
• The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization
• GXJoin: Generalized Cell Transformations for Explainable Joinability
• Concepts, Properties and an Approach for Compositional Generalization
• Sequence-to-Sequence Learning with Latent Neural Grammars
• Learning Algebraic Recombination for Compositional Generalization