Compositional Languages Emerge in a Neural Iterated Learning Model

Abstract

The principle of compositionality, which enables natural language to represent complex concepts via a structured combination of simpler ones, allows us to convey an open-ended set of messages using a limited vocabulary. If compositionality is indeed a natural property of language, we may expect it to appear in communication protocols that are created by neural agents in language games. In this paper, we propose an effective neural iterated learning (NIL) algorithm that, when applied to interacting neural agents, facilitates the emergence of a more structured type of language. Indeed, these languages provide learning speed advantages to neural agents during training, which can be incrementally amplified via NIL. We provide a probabilistic model of NIL and an explanation of why the advantage of compositional language exist. Our experiments confirm our analysis, and also demonstrate that the emerged languages largely improve the generalizing power of the neural agent communication.

accepted by ICLR-2020

References in corpus (3)

• Emergence of Grounded Compositional Language in Multi-Agent Populations
• Emergence of Compositional Language with Deep Generational Transmission
• Measuring Compositionality in Representation Learning

Cited by in corpus (16)

• Emergent Language: A Survey and Taxonomy
• Dialog without Dialog Data: Learning Visual Dialog Agents from VQA Data
• Learning to Draw: Emergent Communication through Sketching
• Emergent Graphical Conventions in a Visual Communication Game
• Countering Language Drift with Seeded Iterated Learning
• Inductive Bias and Language Expressivity in Emergent Communication
• Cultural evolution via iterated learning and communication explains efficient color naming systems
• Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
• Multi-label Iterated Learning for Image Classification with Label Ambiguity
• Iterated learning for emergent systematicity in VQA
• Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictability
• Zero-Shot Generalization using Intrinsically Motivated Compositional Emergent Protocols
• Compositionality Through Language Transmission, using Artificial Neural Networks
• The Emergence of the Shape Bias Results from Communicative Efficiency
• Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT
• TexRel: a Green Family of Datasets for Emergent Communications on Relations