Multi-Agent Cooperation and the Emergence of (Natural) Language

Abstract

The current mainstream approach to train natural language systems is to expose them to large amounts of text. This passive learning is problematic if we are interested in developing interactive machines, such as conversational agents. We propose a framework for language learning that relies on multi-agent communication. We study this learning in the context of referential games. In these games, a sender and a receiver see a pair of images. The sender is told one of them is the target and is allowed to send a message from a fixed, arbitrary vocabulary to the receiver. The receiver must rely on this message to identify the target. Thus, the agents develop their own language interactively out of the need to communicate. We show that two networks with simple configurations are able to learn to coordinate in the referential game. We further explore how to make changes to the game environment to cause the "word meanings" induced in the game to better reflect intuitive semantic properties of the images. In addition, we present a simple strategy for grounding the agents' code into natural language. Both of these are necessary steps towards developing machines that are able to communicate with humans productively.

arXiv commentAccepted at ICLR 2017

Cited by in corpus (49)

• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Anti-efficient encoding in emergent communication
• RODE: Learning Roles to Decompose Multi-Agent Tasks
• Compositional Obverter Communication Learning From Raw Visual Input
• Off-Policy Multi-Agent Decomposed Policy Gradients
• Semantics-Native Communication with Contextual Reasoning
• Emergence of Compositional Language with Deep Generational Transmission
• Emergent Multi-Agent Communication in the Deep Learning Era
• On the interaction between supervision and self-play in emergent communication
• Emergence of Communication in an Interactive World with Consistent Speakers
• Measuring Compositionality in Representation Learning
• Learning Nearly Decomposable Value Functions Via Communication Minimization
• Collaborative Visual Navigation
• Emergence of Pragmatics from Referential Game between Theory of Mind Agents
• Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity
• Emergence of Theory of Mind Collaboration in Multiagent Systems
• Emergent Language: A Survey and Taxonomy
• Interpretable agent communication from scratch (with a generic visual processor emerging on the side)
• Emergent Communication of Generalizations
• Learning to Communicate Implicitly By Actions
• Emergent Discrete Communication in Semantic Spaces
• Quasi-Equivalence Discovery for Zero-Shot Emergent Communication
• Adversarially Guided Self-Play for Adopting Social Conventions
• Incorporating Pragmatic Reasoning Communication into Emergent Language
• Mastering emergent language: learning to guide in simulated navigation
• Learning to Draw: Emergent Communication through Sketching
• Emergent Graphical Conventions in a Visual Communication Game
• Countering Language Drift with Seeded Iterated Learning
• Inductive Bias and Language Expressivity in Emergent Communication
• Avoiding hashing and encouraging visual semantics in referential emergent language games
• Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
• Dynamic population-based meta-learning for multi-agent communication with natural language
• Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?
• Attentional Policies for Cross-Context Multi-Agent Reinforcement Learning
• Modeling Conceptual Understanding in Image Reference Games
• On the Role of Collective Sensing and Evolution in Group Formation
• Symbol Emergence as an Interpersonal Multimodal Categorization
• Towards Graph Representation Learning in Emergent Communication
• Winning an Election: On Emergent Strategic Communication in Multi-Agent Networks
• Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictability
• Paying Attention to Function Words
• Focus on What's Informative and Ignore What's not: Communication Strategies in a Referential Game
• A Signal-Centric Perspective on the Evolution of Symbolic Communication
• Multi-Agent Intention Sharing via Leader-Follower Forest
• Emergence of Writing Systems Through Multi-Agent Cooperation
• Supervised Seeded Iterated Learning for Interactive Language Learning
• Shared Visual Representations of Drawing for Communication: How do different biases affect human interpretability and intent?
• The Emergence of the Shape Bias Results from Communicative Efficiency
• Calibrate your listeners! Robust communication-based training for pragmatic speakers