MortonNet: Self-Supervised Learning of Local Features in 3D Point Clouds

Abstract

We present a self-supervised task on point clouds, in order to learn meaningful point-wise features that encode local structure around each point. Our self-supervised network, named MortonNet, operates directly on unstructured/unordered point clouds. Using a multi-layer RNN, MortonNet predicts the next point in a point sequence created by a popular and fast Space Filling Curve, the Morton-order curve. The final RNN state (coined Morton feature) is versatile and can be used in generic 3D tasks on point clouds. In fact, we show how Morton features can be used to significantly improve performance (+3% for 2 popular semantic segmentation algorithms) in the task of semantic segmentation of point clouds on the challenging and large-scale S3DIS dataset. We also show how MortonNet trained on S3DIS transfers well to another large-scale dataset, vKITTI, leading to an improvement over state-of-the-art of 3.8%. Finally, we use Morton features to train a much simpler and more stable model for part segmentation in ShapeNet. Our results show how our self-supervised task results in features that are useful for 3D segmentation tasks, and generalize well to other datasets.

References in corpus (6)

• Inductive Representation Learning on Large Graphs
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Dynamic Graph CNN for Learning on Point Clouds
• Spatially-sparse convolutional neural networks
• Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55
• Sparse 3D convolutional neural networks

Cited by in corpus (8)

• Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning
• Self-Supervised Learning for Domain Adaptation on Point-Clouds
• Multi-Path Region Mining For Weakly Supervised 3D Semantic Segmentation on Point Clouds
• DAR-Net: Dynamic Aggregation Network for Semantic Scene Segmentation
• Self-supervised Modal and View Invariant Feature Learning
• A Learnable Self-supervised Task for Unsupervised Domain Adaptation on Point Clouds
• Self-supervised Feature Learning by Cross-modality and Cross-view Correspondences
• Self-supervised Point Set Local Descriptors for Point Cloud Registration