Deep Learning for 3D Point Clouds: A Survey

Abstract

Point cloud learning has lately attracted increasing attention due to its wide applications in many areas, such as computer vision, autonomous driving, and robotics. As a dominating technique in AI, deep learning has been successfully used to solve various 2D vision problems. However, deep learning on point clouds is still in its infancy due to the unique challenges faced by the processing of point clouds with deep neural networks. Recently, deep learning on point clouds has become even thriving, with numerous methods being proposed to address different problems in this area. To stimulate future research, this paper presents a comprehensive review of recent progress in deep learning methods for point clouds. It covers three major tasks, including 3D shape classification, 3D object detection and tracking, and 3D point cloud segmentation. It also presents comparative results on several publicly available datasets, together with insightful observations and inspiring future research directions.

arXiv commentAccepted by IEEE TPAMI. Project page: https://github.com/QingyongHu/SoTA-Point-Cloud

References in corpus (38)

• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Modern hierarchical, agglomerative clustering algorithms
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation
• Semantic Instance Segmentation with a Discriminative Loss Function
• Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation
• Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways
• Vehicle Detection from 3D Lidar Using Fully Convolutional Network
• Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds
• HDNET: Exploiting HD Maps for 3D Object Detection
• Spatially-sparse convolutional neural networks
• Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection
• Linked Dynamic Graph CNN: Learning on Point Cloud via Linking Hierarchical Features
• IPOD: Intensive Point-based Object Detector for Point Cloud
• From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network
• RGCNN: Regularized Graph CNN for Point Cloud Segmentation
• PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing
• MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation
• PointDAN: A Multi-Scale 3D Domain Adaption Network for Point Cloud Representation
• 3D-BEVIS: Bird's-Eye-View Instance Segmentation
• PointRGCN: Graph Convolution Networks for 3D Vehicles Detection Refinement
• 3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks
• Efficient Bird Eye View Proposals for 3D Siamese Tracking
• Patch Refinement -- Localized 3D Object Detection
• Noise-resistant Deep Learning for Object Classification in 3D Point Clouds Using a Point Pair Descriptor
• LSANet: Feature Learning on Point Sets by Local Spatial Aware Layer
• Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots
• Deep RBFNet: Point Cloud Feature Learning using Radial Basis Functions
• Learning to Group: A Bottom-Up Framework for 3D Part Discovery in Unseen Categories
• Spherical Convolutional Neural Network for 3D Point Clouds
• DAR-Net: Dynamic Aggregation Network for Semantic Scene Segmentation
• 3DTI-Net: Learn Inner Transform Invariant 3D Geometry Features using Dynamic GCN
• Point Cloud Instance Segmentation using Probabilistic Embeddings
• AdaCoSeg: Adaptive Shape Co-Segmentation with Group Consistency Loss
• PointAtrousGraph: Deep Hierarchical Encoder-Decoder with Point Atrous Convolution for Unorganized 3D Points

Cited by in corpus (13)

• Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks
• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• Geometric Back-projection Network for Point Cloud Classification
• ChartPointFlow for Topology-Aware 3D Point Cloud Generation
• Equivariant Maps for Hierarchical Structures
• Global Context Aware Convolutions for 3D Point Cloud Understanding
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
• DV-ConvNet: Fully Convolutional Deep Learning on Point Clouds with Dynamic Voxelization and 3D Group Convolution
• Learning Scene Dynamics from Point Cloud Sequences
• Unsupervised Object Detection with LiDAR Clues
• A Convolutional Architecture for 3D Model Embedding
• Video-based Point Cloud Compression Artifact Removal
• AccSS3D: Accelerator for Spatially Sparse 3D DNNs