Learning Latent Representations to Influence Multi-Agent Interaction
arXiv:2011.06619
Abstract
Seamlessly interacting with humans or robots is hard because these agents are non-stationary. They update their policy in response to the ego agent's behavior, and the ego agent must anticipate these changes to co-adapt. Inspired by humans, we recognize that robots do not need to explicitly model every low-level action another agent will make; instead, we can capture the latent strategy of other agents through high-level representations. We propose a reinforcement learning-based framework for learning latent representations of an agent's policy, where the ego agent identifies the relationship between its behavior and the other agent's future strategy. The ego agent then leverages these latent dynamics to influence the other agent, purposely guiding them towards policies suitable for co-adaptation. Across several simulated domains and a real-world air hockey game, our approach outperforms the alternatives and learns to influence the other agent.
Conference on Robot Learning (CoRL) 2020. Supplementary website at https://sites.google.com/view/latent-strategies/
References in corpus (15)
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
- Counterfactual Multi-Agent Policy Gradients
- Learning Latent Dynamics for Planning from Pixels
- CARLA: An Open Urban Driving Simulator
- Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
- Emergence of Grounded Compositional Language in Multi-Agent Populations
- Learning Policy Representations in Multiagent Systems
- Machine Theory of Mind
- Modeling Others using Oneself in Multi-Agent Reinforcement Learning
- Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
- Deep Reinforcement Learning amidst Lifelong Non-Stationarity
- E-HBA: Using Action Policies for Expert Advice and Agent Typification
- Variational Autoencoders for Opponent Modeling in Multi-Agent Systems