Learning to Play Guess Who? and Inventing a Grounded Language as a Consequence

Abstract

Acquiring your first language is an incredible feat and not easily duplicated. Learning to communicate using nothing but a few pictureless books, a corpus, would likely be impossible even for humans. Nevertheless, this is the dominating approach in most natural language processing today. As an alternative, we propose the use of situated interactions between agents as a driving force for communication, and the framework of Deep Recurrent Q-Networks for evolving a shared language grounded in the provided environment. We task the agents with interactive image search in the form of the game Guess Who?. The images from the game provide a non trivial environment for the agents to discuss and a natural grounding for the concepts they decide to encode in their communication. Our experiments show that the agents learn not only to encode physical concepts in their words, i.e. grounding, but also that the agents learn to hold a multi-step dialogue remembering the state of the dialogue from step to step.

arXiv commentPrevious version was accepted to Deep Reinforcement Learning Workshop at NIPS 2016

References in corpus (4)

• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Learning Multiagent Communication with Backpropagation
• Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks

Cited by in corpus (20)

• QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning
• Maintaining cooperation in complex social dilemmas using deep reinforcement learning
• Compositional Obverter Communication Learning From Raw Visual Input
• Multi-Agent Common Knowledge Reinforcement Learning
• Emergent Translation in Multi-Agent Communication
• Emergent Multi-Agent Communication in the Deep Learning Era
• Emergent Communication in a Multi-Modal, Multi-Step Referential Game
• Consequentialist conditional cooperation in social dilemmas with imperfect information
• Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity
• How agents see things: On visual representations in an emergent language game
• Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence
• Learning to Communicate in Multi-Agent Reinforcement Learning : A Review
• Adversarially Guided Self-Play for Adopting Social Conventions
• Mastering emergent language: learning to guide in simulated navigation
• Learning to Draw: Emergent Communication through Sketching
• Modeling Conceptual Understanding in Image Reference Games
• Iterated learning for emergent systematicity in VQA
• Multi-agent Policy Optimization with Approximatively Synchronous Advantage Estimation
• PatchGame: Learning to Signal Mid-level Patches in Referential Games