Learning Attentional Communication for Multi-Agent Cooperation

Abstract

Communication could potentially be an effective way for multi-agent cooperation. However, information sharing among all agents or in predefined communication architectures that existing methods adopt can be problematic. When there is a large number of agents, agents cannot differentiate valuable information that helps cooperative decision making from globally shared information. Therefore, communication barely helps, and could even impair the learning of multi-agent cooperation. Predefined communication architectures, on the other hand, restrict communication among agents and thus restrain potential cooperation. To tackle these difficulties, in this paper, we propose an attentional communication model that learns when communication is needed and how to integrate shared information for cooperative decision making. Our model leads to efficient and effective communication for large-scale multi-agent cooperation. Empirically, we show the strength of our model in a variety of cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies than existing methods.

arXiv commentNIPS'18

References in corpus (10)

• Continuous control with deep reinforcement learning
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Recurrent Models of Visual Attention
• Counterfactual Multi-Agent Policy Gradients
• Mean Field Multi-Agent Reinforcement Learning
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• Emergence of Grounded Compositional Language in Multi-Agent Populations
• Playing FPS Games with Deep Reinforcement Learning
• Coordinated Multi-Agent Imitation Learning
• Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning

Cited by in corpus (47)

• Actor-Attention-Critic for Multi-Agent Reinforcement Learning
• Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence
• Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning
• Learning to Schedule Communication in Multi-agent Reinforcement Learning
• Learning Efficient Multi-agent Communication: An Information Bottleneck Approach
• Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning
• Graph Convolutional Reinforcement Learning
• Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learning
• ROMA: Multi-Agent Reinforcement Learning with Emergent Roles
• Succinct and Robust Multi-Agent Communication With Temporal Message Control
• Learning Nearly Decomposable Value Functions Via Communication Minimization
• Counterfactual Multi-Agent Reinforcement Learning with Graph Convolution Communication
• Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradients
• Learning Individually Inferred Communication for Multi-Agent Cooperation
• Learning Distilled Collaboration Graph for Multi-Agent Perception
• Hierarchical Reinforcement Learning for Multi-agent MOBA Game
• Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning
• Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning
• Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning
• Model-based Reinforcement Learning for Service Mesh Fault Resiliency in a Web Application-level
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning
• Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling
• Fairness-aware Competitive Bidding Influence Maximization in Social Networks
• Learning Multi-agent Communication under Limited-bandwidth Restriction for Internet Packet Routing
• Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learning
• Towards Heterogeneous Multi-Agent Reinforcement Learning with Graph Neural Networks
• Multi-Agent Actor-Critic with Hierarchical Graph Attention Network
• From Few to More: Large-scale Dynamic Multiagent Curriculum Learning
• Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance
• Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning
• Agent-Centric Representations for Multi-Agent Reinforcement Learning
• Value-Decomposition Multi-Agent Actor-Critics
• Learning to Communicate Using Counterfactual Reasoning
• Event-Triggered Multi-agent Reinforcement Learning with Communication under Limited-bandwidth Constraint
• Attentional Policies for Cross-Context Multi-Agent Reinforcement Learning
• Correcting Experience Replay for Multi-Agent Communication
• Learning Efficient Multi-Agent Cooperative Visual Exploration
• Human and Multi-Agent collaboration in a human-MARL teaming framework
• Multi-agent Cooperative Games Using Belief Map Assisted Training
• Who2com: Collaborative Perception via Learnable Handshake Communication
• PAC Guarantees for Cooperative Multi-Agent Reinforcement Learning with Restricted Communication
• KnowSR: Knowledge Sharing among Homogeneous Agents in Multi-agent Reinforcement Learning
• Hierarchical RNNs-Based Transformers MADDPG for Mixed Cooperative-Competitive Environments
• Learning Meta Representations for Agents in Multi-Agent Reinforcement Learning
• FireCommander: An Interactive, Probabilistic Multi-agent Environment for Heterogeneous Robot Teams
• Reinforced Natural Language Interfaces via Entropy Decomposition
• DeCOM: Decomposed Policy for Constrained Cooperative Multi-Agent Reinforcement Learning