Learning Multiagent Communication with Backpropagation

Abstract

Many tasks in AI require the collaboration of multiple agents. Typically, the communication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks. The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to learn to communicate amongst themselves, yielding improved performance over non-communicative agents and baselines. In some cases, it is possible to interpret the language devised by the agents, revealing simple but effective strategies for solving the task at hand.

Accepted to NIPS 2016

References in corpus (5)

• Dynamic Memory Networks for Visual and Textual Question Answering
• Neural GPUs Learn Algorithms
• Move Evaluation in Go Using Deep Convolutional Neural Networks
• Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks
• MazeBase: A Sandbox for Learning from Games

Cited by in corpus (76)

• Geometric deep learning: going beyond Euclidean data
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Deep Reinforcement Learning: An Overview
• Few-Shot Learning with Graph Neural Networks
• Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• Residual Gated Graph ConvNets
• The Hanabi Challenge: A New Frontier for AI Research
• A Deep Reinforcement Learning Chatbot
• A Survey on Traffic Signal Control Methods
• A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning
• Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
• MR-GNN: Multi-Resolution and Dual Graph Neural Network for Predicting Structured Entity Interactions
• Learning to Protect Communications with Adversarial Neural Cryptography
• SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
• Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks
• Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning
• The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey
• Modeling Others using Oneself in Multi-Agent Reinforcement Learning
• Learning to Schedule Communication in Multi-agent Reinforcement Learning
• Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
• Graph Partition Neural Networks for Semi-Supervised Classification
• Emergent Translation in Multi-Agent Communication
• Improving Coordination in Small-Scale Multi-Agent Deep Reinforcement Learning through Memory-driven Communication
• Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning
• Learning to Play Guess Who? and Inventing a Grounded Language as a Consequence
• Geometric deep learning on graphs and manifolds using mixture model CNNs
• Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learning
• VAIN: Attentional Multi-agent Predictive Modeling
• AI Research Considerations for Human Existential Safety (ARCHES)
• Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning
• A Paradigm for Situated and Goal-Driven Language Learning
• Combining Strategic Learning and Tactical Search in Real-Time Strategy Games
• Multi-agent Deep Reinforcement Learning with Extremely Noisy Observations
• Message Passing Multi-Agent GANs
• Learning Distilled Collaboration Graph for Multi-Agent Perception
• Quantifying the Alignment of Graph and Features in Deep Learning
• BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
• Hierarchical Reinforcement Learning for Multi-agent MOBA Game
• Factorised Neural Relational Inference for Multi-Interaction Systems
• Learning Multi-Agent Coordination through Connectivity-driven Communication
• Learning Multi-agent Communication under Limited-bandwidth Restriction for Internet Packet Routing
• Learning Agent Communication under Limited Bandwidth by Message Pruning
• GraphTSNE: A Visualization Technique for Graph-Structured Data
• Parameter Sharing Reinforcement Learning Architecture for Multi Agent Driving Behaviors
• Emergent Language: A Survey and Taxonomy
• Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence
• Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance
• Learning Autocomplete Systems as a Communication Game
• Learning to Communicate in Multi-Agent Reinforcement Learning : A Review
• Are Graph Neural Networks Miscalibrated?
• Learning to Communicate Implicitly By Actions
• Reinforcement Communication Learning in Different Social Network Structures
• Mixed Cooperative-Competitive Communication Using Multi-Agent Reinforcement Learning
• StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learning
• Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game
• Analysing Factorizations of Action-Value Networks for Cooperative Multi-Agent Reinforcement Learning
• On the Role of Collective Sensing and Evolution in Group Formation
• Deep learning reveals hidden interactions in complex systems
• Inter-Level Cooperation in Hierarchical Reinforcement Learning
• Analogs of Linguistic Structure in Deep Representations
• Learning Complex Multi-Agent Policies in Presence of an Adversary
• Multi-agent Cooperative Games Using Belief Map Assisted Training
• Structural Inductive Biases in Emergent Communication
• Learning to Communicate with Reinforcement Learning for an Adaptive Traffic Control System
• HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism
• Bi-level Actor-Critic for Multi-agent Coordination
• Learning Actor Relation Graphs for Group Activity Recognition
• Collaborative Deep Reinforcement Learning for Joint Object Search
• Correlation-aware Unsupervised Change-point Detection via Graph Neural Networks
• Planning, Inference and Pragmatics in Sequential Language Games
• Cooperative Multi-Agent Policy Gradients with Sub-optimal Demonstration
• Multi-Agent Imitation Learning with Copulas
• Agent Probing Interaction Policies