Collaborative Visual Navigation

Abstract

As a fundamental problem for Artificial Intelligence, multi-agent system (MAS) is making rapid progress, mainly driven by multi-agent reinforcement learning (MARL) techniques. However, previous MARL methods largely focused on grid-world like or game environments; MAS in visually rich environments has remained less explored. To narrow this gap and emphasize the crucial role of perception in MAS, we propose a large-scale 3D dataset, CollaVN, for multi-agent visual navigation (MAVN). In CollaVN, multiple agents are entailed to cooperatively navigate across photo-realistic environments to reach target locations. Diverse MAVN variants are explored to make our problem more general. Moreover, a memory-augmented communication framework is proposed. Each agent is equipped with a private, external memory to persistently store communication information. This allows agents to make better use of their past communication information, enabling more efficient collaboration and robust long-term planning. In our experiments, several baselines and evaluation metrics are designed. We also empirically verify the efficacy of our proposed MARL approach across different MAVN task settings.

References in corpus (8)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Learning to Explore using Active Neural SLAM
• Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?
• Multi-Agent Cooperation and the Emergence of (Natural) Language
• HoME: a Household Multimodal Environment
• Learning to Schedule Communication in Multi-agent Reinforcement Learning
• PyRobot: An Open-source Robotics Framework for Research and Benchmarking
• Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation