Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks

Abstract

We propose deep distributed recurrent Q-networks (DDRQN), which enable teams of agents to learn to solve communication-based coordination tasks. In these tasks, the agents are not given any pre-designed communication protocol. Therefore, in order to successfully communicate, they must first automatically develop and agree upon their own communication protocol. We present empirical results on two multi-agent learning problems based on well-known riddles, demonstrating that DDRQN can successfully solve such tasks and discover elegant communication protocols to do so. To our knowledge, this is the first time deep reinforcement learning has succeeded in learning communication protocols. In addition, we present ablation experiments that confirm that each of the main components of the DDRQN architecture are critical to its success.

References in corpus (4)

• Dueling Network Architectures for Deep Reinforcement Learning
• Massively Parallel Methods for Deep Reinforcement Learning
• Move Evaluation in Go Using Deep Convolutional Neural Networks
• Data-Efficient Learning of Feedback Policies from Image Pixels using Deep Dynamical Models

Cited by in corpus (28)

• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Learning Multiagent Communication with Backpropagation
• Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
• Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
• Opponent Modeling in Deep Reinforcement Learning
• Modeling Others using Oneself in Multi-Agent Reinforcement Learning
• R-MADDPG for Partially Observable Environments and Limited Communication
• Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks
• Learning to Play Guess Who? and Inventing a Grounded Language as a Consequence
• Emergent Communication in a Multi-Modal, Multi-Step Referential Game
• Mastering Complex Control in MOBA Games with Deep Reinforcement Learning
• Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations
• A Paradigm for Situated and Goal-Driven Language Learning
• Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
• Emergence of Theory of Mind Collaboration in Multiagent Systems
• Likelihood Quantile Networks for Coordinating Multi-Agent Reinforcement Learning
• Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey
• Learning to Communicate in Multi-Agent Reinforcement Learning : A Review
• Incorporating Pragmatic Reasoning Communication into Emergent Language
• Reinforcement Communication Learning in Different Social Network Structures
• Multi-Agent Deep Reinforcement Learning with Adaptive Policies
• On the Role of Collective Sensing and Evolution in Group Formation
• Cell Selection with Deep Reinforcement Learning in Sparse Mobile Crowdsensing
• The red one!: On learning to refer to things based on their discriminative properties
• Inductive-bias-driven Reinforcement Learning For Efficient Schedules in Heterogeneous Clusters
• Building Intelligent Autonomous Navigation Agents
• Towards Brain-inspired System: Deep Recurrent Reinforcement Learning for Simulated Self-driving Agent