MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation

Abstract

We propose a new approach for 3D instance segmentation based on sparse convolution and point affinity prediction, which indicates the likelihood of two points belonging to the same instance. The proposed network, built upon submanifold sparse convolution [3], processes a voxelized point cloud and predicts semantic scores for each occupied voxel as well as the affinity between neighboring voxels at different scales. A simple yet effective clustering algorithm segments points into instances based on the predicted affinity and the mesh topology. The semantic for each instance is determined by the semantic prediction. Experiments show that our method outperforms the state-of-the-art instance segmentation methods by a large margin on the widely used ScanNet benchmark [2]. We share our code publicly at https://github.com/art-programmer/MASC.

References in corpus (6)

• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Submanifold Sparse Convolutional Networks
• Recurrent Slice Networks for 3D Segmentation of Point Clouds
• GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud
• 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans

Cited by in corpus (19)

• Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds
• Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey
• Deep Learning for 3D Point Clouds: A Survey
• Deep Learning Based 3D Segmentation: A Survey
• PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation
• 3D Instance Segmentation of MVS Buildings
• Identifying Unknown Instances for Autonomous Driving
• PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things
• Spatial Semantic Embedding Network: Fast 3D Instance Segmentation with Deep Metric Learning
• DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution
• RevealNet: Seeing Behind Objects in RGB-D Scans
• OccuSeg: Occupancy-aware 3D Instance Segmentation
• Multi-view PointNet for 3D Scene Understanding
• HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor
• Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans
• SASO: Joint 3D Semantic-Instance Segmentation via Multi-scale Semantic Association and Salient Point Clustering Optimization
• Dynamic Convolution for 3D Point Cloud Instance Segmentation
• Point Cloud Instance Segmentation with Semi-supervised Bounding-Box Mining
• Learning to Reconstruct and Segment 3D Objects