Concepts, Properties and an Approach for Compositional Generalization

Abstract

Compositional generalization is the capacity to recognize and imagine a large amount of novel combinations from known components. It is a key in human intelligence, but current neural networks generally lack such ability. This report connects a series of our work for compositional generalization, and summarizes an approach. The first part contains concepts and properties. The second part looks into a machine learning approach. The approach uses architecture design and regularization to regulate information of representations. This report focuses on basic ideas with intuitive and illustrative explanations. We hope this work would be helpful to clarify fundamentals of compositional generalization and lead to advance artificial intelligence.

References in corpus (13)

• Opening the Black Box of Deep Neural Networks via Information
• Deep Learning of Representations: Looking Forward
• The Consciousness Prior
• Compositional generalization in a deep seq2seq model by separating syntax and semantics
• A Benchmark for Systematic Generalization in Grounded Language Understanding
• Recurrent Independent Mechanisms
• Compositional generalization through meta sequence-to-sequence learning
• Compositional Generalization by Learning Analytical Expressions
• Locality and compositionality in zero-shot learning
• Learning to Recombine and Resample Data for Compositional Generalization
• Compositional Generalization for Primitive Substitutions
• Visually Grounded Continual Learning of Compositional Phrases
• Posing Fair Generalization Tasks for Natural Language Inference