Model-based Reinforcement Learning for Decentralized Multiagent Rendezvous
arXiv:2003.06906
Abstract
Collaboration requires agents to align their goals on the fly. Underlying the human ability to align goals with other agents is their ability to predict the intentions of others and actively update their own plans. We propose hierarchical predictive planning (HPP), a model-based reinforcement learning method for decentralized multiagent rendezvous. Starting with pretrained, single-agent point to point navigation policies and using noisy, high-dimensional sensor inputs like lidar, we first learn via self-supervision motion predictions of all agents on the team. Next, HPP uses the prediction models to propose and evaluate navigation subgoals for completing the rendezvous task without explicit communication among agents. We evaluate HPP in a suite of unseen environments, with increasing complexity and numbers of obstacles. We show that HPP outperforms alternative reinforcement learning, path planning, and heuristic-based baselines on challenging, unseen environments. Experiments in the real world demonstrate successful transfer of the prediction models from sim to real world without any additional fine-tuning. Altogether, HPP removes the need for a centralized operator in multiagent systems by combining model-based RL and inference methods, enabling agents to dynamically align plans.
CoRL 2020. The video is available at: https://youtu.be/-ydXHUtPzWE
References in corpus (8)
- Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
- Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
- Collision Avoidance in Pedestrian-Rich Environments with Deep Reinforcement Learning
- Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion
- Machine Theory of Mind
- R-MADDPG for Partially Observable Environments and Limited Communication
- Option Discovery in Hierarchical Reinforcement Learning using Spatio-Temporal Clustering
- Efficient Hierarchical Robot Motion Planning Under Uncertainty and Hybrid Dynamics