Learning Individually Inferred Communication for Multi-Agent Cooperation
arXiv:2006.06455
Abstract
Communication lays the foundation for human cooperation. It is also crucial for multi-agent cooperation. However, existing work focuses on broadcast communication, which is not only impractical but also leads to information redundancy that could even impair the learning process. To tackle these difficulties, we propose Individually Inferred Communication (I2C), a simple yet effective model to enable agents to learn a prior for agent-agent communication. The prior knowledge is learned via causal inference and realized by a feed-forward neural network that maps the agent's local observation to a belief about who to communicate with. The influence of one agent on another is inferred via the joint action-value function in multi-agent reinforcement learning and quantified to label the necessity of agent-agent communication. Furthermore, the agent policy is regularized to better exploit communicated messages. Empirically, we show that I2C can not only reduce communication overhead but also improve the performance in a variety of multi-agent cooperative scenarios, comparing to existing methods. The code is available at https://github.com/PKU-AI-Edge/I2C.
NeurIPS 2020, oral presentation. The code is available at https://github.com/PKU-AI-Edge/I2C
References in corpus (6)
- Recurrent Models of Visual Attention
- Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
- Learning Attentional Communication for Multi-Agent Cooperation
- Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
- QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning
- Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control
Cited by in corpus (4)
- Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance
- Credit Assignment with Meta-Policy Gradient for Multi-Agent Reinforcement Learning
- Learning to Communicate Using Counterfactual Reasoning
- DeCOM: Decomposed Policy for Constrained Cooperative Multi-Agent Reinforcement Learning