Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning

Abstract

We propose spatial semantic embedding network (SSEN), a simple, yet efficient algorithm for 3D instance segmentation using deep metric learning. The raw 3D reconstruction of an indoor environment suffers from occlusions, noise, and is produced without any meaningful distinction between individual entities. For high-level intelligent tasks from a large scale scene, 3D instance segmentation recognizes individual instances of objects. We approach the instance segmentation by simply learning the correct embedding space that maps individual instances of objects into distinct clusters that reflect both spatial and semantic information. Unlike previous approaches that require complex pre-processing or post-processing, our implementation is compact and fast with competitive performance, maintaining scalability on large scenes with high resolution voxels. We demonstrate the state-of-the-art performance of our algorithm in the ScanNet 3D instance segmentation benchmark on AP score.

References in corpus (7)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Semantic Instance Segmentation with a Discriminative Loss Function
• Semantic Instance Segmentation via Deep Metric Learning
• MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation
• GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud
• PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things

Cited by in corpus (4)

• STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos
• LiDAR-based Panoptic Segmentation via Dynamic Shifting Network
• Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks
• HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor