Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols

Abstract

Learning to communicate through interaction, rather than relying on explicit supervision, is often considered a prerequisite for developing a general AI. We study a setting where two agents engage in playing a referential game and, from scratch, develop a communication protocol necessary to succeed in this game. Unlike previous work, we require that messages they exchange, both at train and test time, are in the form of a language (i.e. sequences of discrete symbols). We compare a reinforcement learning approach and one using a differentiable relaxation (straight-through Gumbel-softmax estimator) and observe that the latter is much faster to converge and it results in more effective protocols. Interestingly, we also observe that the protocol we induce by optimizing the communication success exhibits a degree of compositionality and variability (i.e. the same information can be phrased in different ways), both properties characteristic of natural languages. As the ultimate goal is to ensure that communication is accomplished in natural language, we also perform experiments where we inject prior information about natural language into our model and study properties of the resulting protocol.

arXiv commentThe paper was accepted at NIPS 2017. The extended abstract was presented at ICLR 2017 workshop track

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Categorical Reparameterization with Gumbel-Softmax
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Microsoft COCO Captions: Data Collection and Evaluation Server
• Learning Multiagent Communication with Backpropagation
• Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
• Memory Augmented Neural Networks with Wormhole Connections
• Language as a Latent Variable: Discrete Generative Models for Sentence Compression

Cited by in corpus (18)

• Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
• Maintaining cooperation in complex social dilemmas using deep reinforcement learning
• Learning to Schedule Communication in Multi-agent Reinforcement Learning
• Emergent Translation in Multi-Agent Communication
• Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks
• Emergent Communication through Negotiation
• ACCNet: Actor-Coordinator-Critic Net for "Learning-to-Communicate" with Deep Multi-agent Reinforcement Learning
• Deep Learning Based Chatbot Models
• Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input
• Emergent Communication in a Multi-Modal, Multi-Step Referential Game
• Consequentialist conditional cooperation in social dilemmas with imperfect information
• Evaluating the capability of large language models to personalize science texts for diverse middle-school-age learners
• How agents see things: On visual representations in an emergent language game
• Emergent Language: A Survey and Taxonomy
• Zero-Resource Neural Machine Translation with Multi-Agent Communication Game
• Compositional properties of emergent languages in deep learning
• A Signal-Centric Perspective on the Evolution of Symbolic Communication
• Seq2Seq Mimic Games: A Signaling Perspective