Deep Learning Based 3D Segmentation: A Survey

Abstract

3D segmentation is a fundamental and challenging problem in computer vision with applications in autonomous driving and robotics. It has received significant attention from the computer vision, graphics and machine learning communities. Conventional methods for 3D segmentation, based on hand-crafted features and machine learning classifiers, lack generalization ability. Driven by their success in 2D computer vision, deep learning techniques have recently become the tool of choice for 3D segmentation tasks. This has led to an influx of many methods in the literature that have been evaluated on different benchmark datasets. Whereas survey papers on RGB-D and point cloud segmentation exist, there is a lack of a recent in-depth survey that covers all 3D data modalities and application domains. This paper fills the gap and comprehensively surveys the recent progress in deep learning-based 3D segmentation techniques. We cover over 220 works from the last six years, analyze their strengths and limitations, and discuss their competitive results on benchmark datasets. The survey provides a summary of the most commonly used pipelines and finally highlights promising research directions for the future.

30 pages, 10 tables, 8 figures, update the segmentation method to 2024, add the segmentation application in semantic map construction and cultural heritage preservation, change the paper format

References in corpus (20)

• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning
• MeshCNN: A Network with an Edge
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation
• PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation
• Indoor Semantic Segmentation using depth information
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• An application of cascaded 3D fully convolutional networks for medical image segmentation
• Building Generalizable Agents with a Realistic and Rich 3D Environment
• Hierarchical 3D fully convolutional networks for multi-organ segmentation
• Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey
• PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud
• MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation
• HoME: a Household Multimodal Environment
• 3D-BEVIS: Bird's-Eye-View Instance Segmentation
• Towards Automatic Abdominal Multi-Organ Segmentation in Dual Energy CT using Cascaded 3D Fully Convolutional Network
• SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation
• Weakly Supervised Semantic Point Cloud Segmentation:Towards 10X Fewer Labels
• Conditional Random Fields as Recurrent Neural Networks for 3D Medical Imaging Segmentation

Cited by in corpus (5)

• Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanics
• Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions
• Generating 3D Bio-Printable Patches Using Wound Segmentation and Reconstruction to Treat Diabetic Foot Ulcers
• NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes
• Scalable 3D Panoptic Segmentation As Superpoint Graph Clustering