Relational Forward Models for Multi-Agent Learning

Abstract

The behavioral dynamics of multi-agent systems have a rich and orderly structure, which can be leveraged to understand these systems, and to improve how artificial agents learn to operate in them. Here we introduce Relational Forward Models (RFM) for multi-agent learning, networks that can learn to make accurate predictions of agents' future behavior in multi-agent environments. Because these models operate on the discrete entities and relations present in the environment, they produce interpretable intermediate representations which offer insights into what drives agents' behavior, and what events mediate the intensity and valence of social interactions. Furthermore, we show that embedding RFM modules inside agents results in faster learning systems compared to non-augmented baselines. As more and more of the autonomous systems we develop and interact with become multi-agent in nature, developing richer analysis tools for characterizing how and why agents make decisions is increasingly necessary. Moreover, developing artificial agents that quickly and safely learn to coordinate with one another, and with humans in shared environments, is crucial.

Cited by in corpus (13)

• Learning to Simulate Complex Physics with Graph Networks
• Kernel Mean Embedding of Distributions: A Review and Beyond
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Recurrent Independent Mechanisms
• Learning Reciprocity in Complex Sequential Social Dilemmas
• Multi-agent Trajectory Prediction with Fuzzy Query Attention
• Action Semantics Network: Considering the Effects of Actions in Multiagent Systems
• Physion: Evaluating Physical Prediction from Vision in Humans and Machines
• A Sufficient Statistic for Influence in Structured Multiagent Environments
• Agent Modelling under Partial Observability for Deep Reinforcement Learning
• Joint Attention for Multi-Agent Coordination and Social Learning
• Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning
• Dynamic Relational Inference in Multi-Agent Trajectories