Emergence of Communication in an Interactive World with Consistent Speakers

Abstract

Training agents to communicate with one another given task-based supervision only has attracted considerable attention recently, due to the growing interest in developing models for human-agent interaction. Prior work on the topic focused on simple environments, where training using policy gradient was feasible despite the non-stationarity of the agents during training. In this paper, we present a more challenging environment for testing the emergence of communication from raw pixels, where training using policy gradient fails. We propose a new model and training algorithm, that utilizes the structure of a learned representation space to produce more consistent speakers at the initial phases of training, which stabilizes learning. We empirically show that our algorithm substantially improves performance compared to policy gradient. We also propose a new alignment-based metric for measuring context-independence in emerged communication and find our method increases context-independence compared to policy gradient and other competitive baselines.

arXiv commentEmergent Communication Workshop @ NeurIPS 2018

References in corpus (7)

• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Grounded Language Learning in a Simulated 3D World
• Multi-Agent Cooperation and the Emergence of (Natural) Language
• Compositional Obverter Communication Learning From Raw Visual Input
• Emergent Communication through Negotiation
• A Paradigm for Situated and Goal-Driven Language Learning
• Towards Multi-Agent Communication-Based Language Learning

Cited by in corpus (14)

• Emergent Multi-Agent Communication in the Deep Learning Era
• Developmentally motivated emergence of compositional communication via template transfer
• Measuring Compositionality in Representation Learning
• Ease-of-Teaching and Language Structure from Emergent Communication
• Emergence of Pragmatics from Referential Game between Theory of Mind Agents
• Towards More Human-like AI Communication: A Review of Emergent Communication Research
• Emergent Language: A Survey and Taxonomy
• Measuring non-trivial compositionality in emergent communication
• Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
• Re-conceptualising the Language Game Paradigm in the Framework of Multi-Agent Reinforcement Learning
• On (Emergent) Systematic Generalisation and Compositionality in Visual Referential Games with Straight-Through Gumbel-Softmax Estimator
• Winning an Election: On Emergent Strategic Communication in Multi-Agent Networks
• On Memory Mechanism in Multi-Agent Reinforcement Learning
• Emergent Communication with World Models