Learning Agent Communication under Limited Bandwidth by Message Pruning
arXiv:1912.05304
Abstract
Communication is a crucial factor for the big multi-agent world to stay organized and productive. Recently, Deep Reinforcement Learning (DRL) has been applied to learn the communication strategy and the control policy for multiple agents. However, the practical \emph{\textbf{limited bandwidth}} in multi-agent communication has been largely ignored by the existing DRL methods. Specifically, many methods keep sending messages incessantly, which consumes too much bandwidth. As a result, they are inapplicable to multi-agent systems with limited bandwidth. To handle this problem, we propose a gating mechanism to adaptively prune less beneficial messages. We evaluate the gating mechanism on several tasks. Experiments demonstrate that it can prune a lot of messages with little impact on performance. In fact, the performance may be greatly improved by pruning redundant messages. Moreover, the proposed gating mechanism is applicable to several previous methods, equipping them the ability to address bandwidth restricted settings.
accepted as a regular paper with poster presentation @ AAAI20. arXiv admin note: text overlap with arXiv:1903.05561
References in corpus (12)
- Continuous control with deep reinforcement learning
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Binarized Neural Networks
- Learning Multiagent Communication with Backpropagation
- Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
- A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
- Winner-Take-All Autoencoders
- Parameter Sharing Deep Deterministic Policy Gradient for Cooperative Multi-agent Reinforcement Learning
- Modelling the Dynamic Joint Policy of Teammates with Attention Multi-agent DDPG
- Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning
- ACCNet: Actor-Coordinator-Critic Net for "Learning-to-Communicate" with Deep Multi-agent Reinforcement Learning
- Multi-agent Deep Reinforcement Learning with Extremely Noisy Observations