OccuSeg: Occupancy-aware 3D Instance Segmentation
arXiv:2003.06537
Abstract
3D instance segmentation, with a variety of applications in robotics and augmented reality, is in large demands these days. Unlike 2D images that are projective observations of the environment, 3D models provide metric reconstruction of the scenes without occlusion or scale ambiguity. In this paper, we define "3D occupancy size", as the number of voxels occupied by each instance. It owns advantages of robustness in prediction, on which basis, OccuSeg, an occupancy-aware 3D instance segmentation scheme is proposed. Our multi-task learning produces both occupancy signal and embedding representations, where the training of spatial and feature embeddings varies with their difference in scale-aware. Our clustering scheme benefits from the reliable comparison between the predicted occupancy size and the clustered occupancy size, which encourages hard samples being correctly clustered and avoids over segmentation. The proposed approach achieves state-of-the-art performance on 3 real-world datasets, i.e. ScanNetV2, S3DIS and SceneNN, while maintaining high efficiency.
CVPR 2020, video this https URL https://youtu.be/co7y6LQ7Kqc
References in corpus (8)
- ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
- Semantic Instance Segmentation with a Discriminative Loss Function
- Semantic Instance Segmentation via Deep Metric Learning
- Spatially-sparse convolutional neural networks
- MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation
- GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud
- Efficient semantic image segmentation with superpixel pooling
- PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things