Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
arXiv:2111.06464
Abstract
Communication is compositional if complex signals can be represented as a combination of simpler subparts. In this paper, we theoretically show that inductive biases on both the training framework and the data are needed to develop a compositional communication. Moreover, we prove that compositionality spontaneously arises in the signaling games, where agents communicate over a noisy channel. We experimentally confirm that a range of noise levels, which depends on the model and the data, indeed promotes compositionality. Finally, we provide a comprehensive study of this dependence and report results in terms of recently studied compositionality metrics: topographical similarity, conflict count, and context independence.
arXiv commentNeurIPS 2021
References in corpus (19)
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Disentangling by Factorising
- Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
- Emergence of Grounded Compositional Language in Multi-Agent Populations
- Multi-Agent Cooperation and the Emergence of (Natural) Language
- Language as an Abstraction for Hierarchical Deep Reinforcement Learning
- Compositional Obverter Communication Learning From Raw Visual Input
- Emergent Translation in Multi-Agent Communication
- Discrete Autoencoders for Sequence Models
- Emergence of Compositional Language with Deep Generational Transmission
- Compositional Languages Emerge in a Neural Iterated Learning Model
- Emergence of Communication in an Interactive World with Consistent Speakers
- Developmentally motivated emergence of compositional communication via template transfer
- Measuring non-trivial compositionality in emergent communication
- Inductive Bias and Language Expressivity in Emergent Communication
- Structural Inductive Biases in Emergent Communication
- Towards Graph Representation Learning in Emergent Communication