Joint Attention for Multi-Agent Coordination and Social Learning

Abstract

Joint attention - the ability to purposefully coordinate attention with another agent, and mutually attend to the same thing -- is a critical component of human social cognition. In this paper, we ask whether joint attention can be useful as a mechanism for improving multi-agent coordination and social learning. We first develop deep reinforcement learning (RL) agents with a recurrent visual attention architecture. We then train agents to minimize the difference between the attention weights that they apply to the environment at each timestep, and the attention of other agents. Our results show that this joint attention incentive improves agents' ability to solve difficult coordination tasks, by reducing the exponential cost of exploring the joint multi-agent action space. Joint attention leads to higher performance than a competitive centralized critic baseline across multiple environments. Further, we show that joint attention enhances agents' ability to learn from experts present in their environment, even when completing hard exploration tasks that do not require coordination. Taken together, these findings suggest that joint attention may be a useful inductive bias for multi-agent learning.

References in corpus (15)

• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Recurrent Models of Visual Attention
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• Multi-agent Reinforcement Learning in Sequential Social Dilemmas
• Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
• Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?
• Assessing Generalization in Deep Reinforcement Learning
• Generalization and Regularization in DQN
• Modeling Others using Oneself in Multi-Agent Reinforcement Learning
• Relational Forward Models for Multi-Agent Learning
• Graph Convolutional Reinforcement Learning
• Multi-focus Attention Network for Efficient Deep Reinforcement Learning
• Learning to Solve Vehicle Routing Problems with Time Windows through Joint Attention
• Translating Math Formula Images to LaTeX Sequences Using Deep Neural Networks with Sequence-level Training