Learning to Share and Hide Intentions using Information Regularization

Abstract

Learning to cooperate with friends and compete with foes is a key component of multi-agent reinforcement learning. Typically to do so, one requires access to either a model of or interaction with the other agent(s). Here we show how to learn effective strategies for cooperation and competition in an asymmetric information game with no such model or interaction. Our approach is to encourage an agent to reveal or hide their intentions using an information-theoretic regularizer. We consider both the mutual information between goal and action given state, as well as the mutual information between goal and state. We show how to optimize these regularizers in a way that is easy to integrate with policy gradient reinforcement learning. Finally, we demonstrate that cooperative (competitive) policies learned with our approach lead to more (less) reward for a second agent in two simple asymmetric information games.

arXiv commentPresented at the 32nd Conference on Neural Information Processing Systems (NIPS 2018)

Cited by in corpus (14)

• InfoBot: Transfer and Exploration via the Information Bottleneck
• Finding Friend and Foe in Multi-Agent Games
• Exploiting Hierarchy for Learning and Transfer in KL-regularized RL
• Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning
• Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning
• Behavior Priors for Efficient Reinforcement Learning
• Learning to Communicate Implicitly By Actions
• Influence-Based Multi-Agent Exploration
• Preventing Imitation Learning with Adversarial Policy Ensembles
• Robust Multi-Agent Reinforcement Learning with Social Empowerment for Coordination and Communication
• IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL
• Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learning
• DAIR: Disentangled Attention Intrinsic Regularization for Safe and Efficient Bimanual Manipulation
• Maximum Entropy Diverse Exploration: Disentangling Maximum Entropy Reinforcement Learning