Robust Multi-Agent Reinforcement Learning with Social Empowerment for Coordination and Communication
arXiv:2012.08255
Abstract
We consider the problem of robust multi-agent reinforcement learning (MARL) for cooperative communication and coordination tasks. MARL agents, mainly those trained in a centralized way, can be brittle because they can adopt policies that act under the expectation that other agents will act a certain way rather than react to their actions. Our objective is to bias the learning process towards finding strategies that remain reactive towards others' behavior. Social empowerment measures the potential influence between agents' actions. We propose it as an additional reward term, so agents better adapt to other agents' actions. We show that the proposed method results in obtaining higher rewards faster and a higher success rate in three cooperative communication and coordination tasks.
References in corpus (10)
- Continuous control with deep reinforcement learning
- Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
- Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
- Large-Scale Study of Curiosity-Driven Learning
- Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
- Variational Intrinsic Control
- "Other-Play" for Zero-Shot Coordination
- Learning to Share and Hide Intentions using Information Regularization
- A unified strategy for implementing curiosity and empowerment driven reinforcement learning
- New And Surprising Ways to Be Mean. Adversarial NPCs with Coupled Empowerment Minimisation