Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Survey

Abstract

Point cloud data have been widely explored due to its superior accuracy and robustness under various adverse situations. Meanwhile, deep neural networks (DNNs) have achieved very impressive success in various applications such as surveillance and autonomous driving. The convergence of point cloud and DNNs has led to many deep point cloud models, largely trained under the supervision of large-scale and densely-labelled point cloud data. Unsupervised point cloud representation learning, which aims to learn general and useful point cloud representations from unlabelled point cloud data, has recently attracted increasing attention due to the constraint in large-scale point cloud labelling. This paper provides a comprehensive review of unsupervised point cloud representation learning using DNNs. It first describes the motivation, general pipelines as well as terminologies of the recent studies. Relevant background including widely adopted point cloud datasets and DNN architectures is then briefly presented. This is followed by an extensive discussion of existing unsupervised point cloud representation learning methods according to their technical approaches. We also quantitatively benchmark and discuss the reviewed methods over multiple widely adopted point cloud datasets. Finally, we share our humble opinion about several challenges and problems that could be pursued in future research in unsupervised point cloud representation learning. A project associated with this survey has been built at https://github.com/xiaoaoran/3d_url_survey.

arXiv commentIEEE Transactions on Pattern Analysis and Machine Intelligence

References in corpus (21)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Bootstrap your own latent: A new approach to self-supervised Learning
• ShapeNet: An Information-Rich 3D Model Repository
• Improved Baselines with Momentum Contrastive Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Unsupervised Representation Learning by Predicting Image Rotations
• Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation
• Deep Clustering for Unsupervised Learning of Visual Features
• One Million Scenes for Autonomous Driving: ONCE Dataset
• Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training
• Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning
• L2G Auto-encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention
• Deep Unsupervised Learning of 3D Point Clouds via Graph Topology Inference and Filtering
• FPS-Net: A Convolutional Fusion Network for Large-Scale LiDAR Point Cloud Segmentation
• TorchSparse: Efficient Point Cloud Inference Engine
• P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting
• NeuralSampler: Euclidean Point Cloud Auto-Encoder and Sampler

Cited by in corpus (8)

• Deep Learning on 3D Semantic Segmentation: A Detailed Review
• Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation
• Point-DAE: Denoising Autoencoders for Self-supervised Point Cloud Learning
• Self-supervised Learning of Rotation-invariant 3D Point Set Features using Transformer and its Self-distillation
• LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Driving
• LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives
• Region-Transformer: Self-Attention Region Based Class-Agnostic Point Cloud Segmentation
• Aligning Instance-Semantic Sparse Representation towards Unsupervised Object Segmentation and Shape Abstraction with Repeatable Primitives