TarMAC: Targeted Multi-Agent Communication

Abstract

We propose a targeted communication architecture for multi-agent reinforcement learning, where agents learn both what messages to send and whom to address them to while performing cooperative tasks in partially-observable environments. This targeting behavior is learnt solely from downstream task-specific reward without any communication supervision. We additionally augment this with a multi-round communication approach where agents coordinate via multiple rounds of communication before taking actions in the environment. We evaluate our approach on a diverse set of cooperative multi-agent tasks, of varying difficulties, with varying number of agents, in a variety of environments ranging from 2D grid layouts of shapes and simulated traffic junctions to 3D indoor environments, and demonstrate the benefits of targeted and multi-round communication. Moreover, we show that the targeted communication strategies learned by agents are interpretable and intuitive. Finally, we show that our architecture can be easily extended to mixed and competitive environments, leading to improved performance and sample complexity over recent state-of-the-art approaches.

ICML 2019

References in corpus (5)

• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
• Counterfactual Multi-Agent Policy Gradients
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks

Cited by in corpus (9)

• Succinct and Robust Multi-Agent Communication With Temporal Message Control
• BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
• OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research
• Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning
• Learning to Communicate Using Counterfactual Reasoning
• Multi-agent Cooperative Games Using Belief Map Assisted Training
• Learning Complex Multi-Agent Policies in Presence of an Adversary
• Scaling Up Multiagent Reinforcement Learning for Robotic Systems: Learn an Adaptive Sparse Communication Graph
• Emergent Communication with World Models