Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control

Abstract

Multi-agent reinforcement learning (MARL) has recently received considerable attention due to its applicability to a wide range of real-world applications. However, achieving efficient communication among agents has always been an overarching problem in MARL. In this work, we propose Variance Based Control (VBC), a simple yet efficient technique to improve communication efficiency in MARL. By limiting the variance of the exchanged messages between agents during the training phase, the noisy component in the messages can be eliminated effectively, while the useful part can be preserved and utilized by the agents for better performance. Our evaluation using a challenging set of StarCraft II benchmarks indicates that our method achieves $2-10\times$ lower in communication overhead than state-of-the-art MARL algorithms, while allowing agents to better collaborate by developing sophisticated strategies.

Cited by in corpus (8)

• QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• Learning Individually Inferred Communication for Multi-Agent Cooperation
• On the Robustness of Cooperative Multi-Agent Reinforcement Learning
• DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learning
• Heterogeneous Graph Attention Networks for Learning Diverse Communication
• Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictability
• Multi-Agent Intention Sharing via Leader-Follower Forest
• FireCommander: An Interactive, Probabilistic Multi-agent Environment for Heterogeneous Robot Teams