Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency

Abstract

For robotic interaction in environments shared with other agents, access to volumetric and semantic maps of the scene is crucial. However, such environments are inevitably subject to long-term changes, which the map needs to account for. We thus propose panoptic multi-TSDFs as a novel representation for multi-resolution volumetric mapping in changing environments. By leveraging high-level information for 3D reconstruction, our proposed system allocates high resolution only where needed. Through reasoning on the object level, semantic consistency over time is achieved. This enables our method to maintain up-to-date reconstructions with high accuracy while improving coverage by incorporating previous data. We show in thorough experimental evaluation that our map can be efficiently constructed, maintained, and queried during online operation, and that the presented approach can operate robustly on real depth sensors using non-optimized panoptic segmentation as input.

Accepted for ICRA 2022. 7 pages, 8 figures. Code: https://github.com/ethz-asl/panoptic_mapping, Video: https://www.youtube.com/watch?v=A7o2Vy7_TV4

References in corpus (7)

• Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery
• Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects
• Voxgraph: Globally Consistent, Volumetric Mapping using Signed Distance Function Submaps
• EM-Fusion: Dynamic Object-Level SLAM with Probabilistic Data Association
• SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks
• A Unified Approach for Autonomous Volumetric Exploration of Large Scale Environments under Severe Odometry Drift
• RigidFusion: Robot Localisation and Mapping in Environments with Large Dynamic Rigid Objects

Cited by in corpus (12)

• From SLAM to Situational Awareness: Challenges and Survey
• 3D VSG: Long-term Semantic Scene Change Prediction through 3D Variable Scene Graphs
• Real-Time Metric-Semantic Mapping for Autonomous Navigation in Outdoor Environments
• Unsupervised Continual Semantic Adaptation through Neural Rendering
• PlaneSDF-based Change Detection for Long-term Dense Mapping
• DualMap: Online Open-Vocabulary Semantic Mapping for Natural Language Navigation in Dynamic Changing Scenes
• Mapping the Unseen: Unified Promptable Panoptic Mapping with Dynamic Labeling using Foundation Models
• Object-Oriented Grid Mapping in Dynamic Environments
• REACT: Real-time Efficient Attribute Clustering and Transfer for Updatable 3D Scene Graph
• Fast and Compute-efficient Sampling-based Local Exploration Planning via Distribution Learning
• Event-Grounding Graph: Unified Spatio-Temporal Scene Graph from Robotic Observations
• Efficient Prediction of Dense Visual Embeddings via Distillation and RGB-D Transformers