Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning
arXiv:2003.13085
Abstract
Multi-agent reinforcement learning is a standard framework for modeling multi-agent interactions applied in real-world scenarios. Inspired by experience sharing in human groups, learning knowledge parallel reusing between agents can potentially promote team learning performance, especially in multi-task environments. When all agents interact with the environment and learn simultaneously, how each independent agent selectively learns from other agents' behavior knowledge is a problem that we need to solve. This paper proposes a novel knowledge transfer framework in MARL, PAT (Parallel Attentional Transfer). We design two acting modes in PAT, student mode and self-learning mode. Each agent in our approach trains a decentralized student actor-critic to determine its acting mode at each time step. When agents are unfamiliar with the environment, the shared attention mechanism in student mode effectively selects learning knowledge from other agents to decide agents' actions. PAT outperforms state-of-the-art empirical evaluation results against the prior advising approaches. Our approach not only significantly improves team learning rate and global performance, but also is flexible and transferable to be applied in various multi-agent systems.
References in corpus (11)
- Continuous control with deep reinforcement learning
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
- Cooperative Inverse Reinforcement Learning
- Learning Attentional Communication for Multi-Agent Cooperation
- Emergence of Grounded Compositional Language in Multi-Agent Populations
- Control of Memory, Active Perception, and Action in Minecraft
- Coordinated Multi-Agent Imitation Learning
- Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer
- Multi-focus Attention Network for Efficient Deep Reinforcement Learning
- TarMAC: Targeted Multi-Agent Communication