RO-MAP: Real-Time Multi-Object Mapping with Neural Radiance Fields

Abstract

Accurate perception of objects in the environment is important for improving the scene understanding capability of SLAM systems. In robotic and augmented reality applications, object maps with semantic and metric information show attractive advantages. In this paper, we present RO-MAP, a novel multi-object mapping pipeline that does not rely on 3D priors. Given only monocular input, we use neural radiance fields to represent objects and couple them with a lightweight object SLAM based on multi-view geometry, to simultaneously localize objects and implicitly learn their dense geometry. We create separate implicit models for each detected object and train them dynamically and in parallel as new observations are added. Experiments on synthetic and real-world datasets demonstrate that our method can generate semantic object map with shape reconstruction, and be competitive with offline methods while achieving real-time performance (25Hz). The code and dataset will be available at: https://github.com/XiaoHan-Git/RO-MAP

arXiv commentThe code and dataset are available at: https://github.com/XiaoHan-Git/RO-MAP

References in corpus (12)

• ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras
• Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
• Extended Isolation Forest
• CubeSLAM: Monocular 3D Object SLAM
• The Replica Dataset: A Digital Replica of Indoor Spaces
• Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation
• EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association
• Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework using Visual and Depth Cues
• iDF-SLAM: End-to-End RGB-D SLAM with Neural Implicit Mapping and Deep Feature Tracking
• ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description
• Implicit Object Mapping With Noisy Data
• Just Flip: Flipped Observation Generation and Optimization for Neural Radiance Fields to Cover Unobserved View