PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding

Abstract

Arguably one of the top success stories of deep learning is transfer learning. The finding that pre-training a network on a rich source set (eg., ImageNet) can help boost performance once fine-tuned on a usually much smaller target set, has been instrumental to many applications in language and vision. Yet, very little is known about its usefulness in 3D point cloud understanding. We see this as an opportunity considering the effort required for annotating data in 3D. In this work, we aim at facilitating research on 3D representation learning. Different from previous works, we focus on high-level scene understanding tasks. To this end, we select a suite of diverse datasets and tasks to measure the effect of unsupervised pre-training on a large source set of 3D scenes. Our findings are extremely encouraging: using a unified triplet of architecture, source dataset, and contrastive loss for pre-training, we achieve improvement over recent best results in segmentation and detection across 6 different benchmarks for indoor and outdoor, real and synthetic datasets -- demonstrating that the learned representation can generalize across domains. Furthermore, the improvement was similar to supervised pre-training, suggesting that future efforts should favor scaling data collection over more detailed annotation. We hope these findings will encourage more research on unsupervised pretext task design for 3D deep learning.

arXiv commentECCV 2020 (Spotlight); code available at https://github.com/facebookresearch/PointContrast

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Learning deep representations by mutual information estimation and maximization
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Contrastive Multiview Coding
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• RGCNN: Regularized Graph CNN for Point Cloud Segmentation
• On Learning Sets of Symmetric Elements
• Spherical Convolutional Neural Network for 3D Point Clouds
• Continuous Geodesic Convolutions for Learning on 3D Shapes

Cited by in corpus (21)

• Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labels
• Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning
• DPFM: Deep Partial Functional Maps
• Dense Contrastive Learning for Self-Supervised Visual Pre-Training
• P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding
• Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges
• Learning from 2D: Contrastive Pixel-to-Point Knowledge Transfer for 3D Pretraining
• Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion
• Label-Efficient Point Cloud Semantic Segmentation: An Active Learning Approach
• Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts
• Unsupervised Point Cloud Pre-Training via Occlusion Completion
• One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation
• Self-Supervised Pillar Motion Learning for Autonomous Driving
• 3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding
• Pri3D: Can 3D Priors Help 2D Representation Learning?
• Staying in Shape: Learning Invariant Shape Representations using Contrastive Learning
• Few-shot 3D Point Cloud Semantic Segmentation
• Self-supervised Geometric Perception
• Point Discriminative Learning for Data-efficient 3D Point Cloud Analysis
• Deeper or Wider Networks of Point Clouds with Self-attention?
• A LiDAR Assisted Control Module with High Precision in Parking Scenarios for Autonomous Driving Vehicle