Associatively Segmenting Instances and Semantics in Point Clouds

Abstract

A 3D point cloud describes the real scene precisely and intuitively.To date how to segment diversified elements in such an informative 3D scene is rarely discussed. In this paper, we first introduce a simple and flexible framework to segment instances and semantics in point clouds simultaneously. Then, we propose two approaches which make the two tasks take advantage of each other, leading to a win-win situation. Specifically, we make instance segmentation benefit from semantic segmentation through learning semantic-aware point-level instance embedding. Meanwhile, semantic features of the points belonging to the same instance are fused together to make more accurate per-point semantic predictions. Our method largely outperforms the state-of-the-art method in 3D instance segmentation along with a significant improvement in 3D semantic segmentation. Code has been made available at: https://github.com/WXinlong/ASIS.

Accepted by CVPR2019

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Learning to Segment Object Candidates
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• Semantic Instance Segmentation with a Discriminative Loss Function

Cited by in corpus (6)

• Learning Gaussian Instance Segmentation in Point Clouds
• PIG-Net: Inception based Deep Learning Architecture for 3D Point Cloud Segmentation
• PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation
• 3D Point Cloud Descriptors in Hand-crafted and Deep Learning Age: State-of-the-Art
• Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion
• 3DCFS: Fast and Robust Joint 3D Semantic-Instance Segmentation via Coupled Feature Selection