MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation

Abstract

Navigation tasks in photorealistic 3D environments are challenging because they require perception and effective planning under partial observability. Recent work shows that map-like memory is useful for long-horizon navigation tasks. However, a focused investigation of the impact of maps on navigation tasks of varying complexity has not yet been performed. We propose the multiON task, which requires navigation to an episode-specific sequence of objects in a realistic environment. MultiON generalizes the ObjectGoal navigation task and explicitly tests the ability of navigation agents to locate previously observed goal objects. We perform a set of multiON experiments to examine how a variety of agent models perform across a spectrum of navigation task complexities. Our experiments show that: i) navigation performance degrades dramatically with escalating task complexity; ii) a simple semantic map agent performs surprisingly well relative to more complex neural image feature map agents; and iii) even oracle map agents achieve relatively low performance, indicating the potential for future work in training embodied navigation agents using maps. Video summary: https://youtu.be/yqTlHNIcgnY

Project page: https://shivanshpatel35.github.io/multi-ON/ ; the first three authors contributed equally

Cited by in corpus (5)

• From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN)
• Deep Learning for Embodied Vision Navigation: A Survey
• Learning to Map for Active Semantic Goal Navigation
• Spot the Difference: A Novel Task for Embodied Agents in Changing Environments
• VSGM -- Enhance robot task understanding ability through visual semantic graph