Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
arXiv:2006.04307
Abstract
3D semantic segmentation is a fundamental task for robotic and autonomous driving applications. Recent works have been focused on using deep learning techniques, whereas developing fine-annotated 3D LiDAR datasets is extremely labor intensive and requires professional skills. The performance limitation caused by insufficient datasets is called data hunger problem. This research provides a comprehensive survey and experimental study on the question: are we hungry for 3D LiDAR data for semantic segmentation? The studies are conducted at three levels. First, a broad review to the main 3D LiDAR datasets is conducted, followed by a statistical analysis on three representative datasets to gain an in-depth view on the datasets' size and diversity, which are the critical factors in learning deep models. Second, a systematic review to the state-of-the-art 3D semantic segmentation is conducted, followed by experiments and cross examinations of three representative deep learning methods to find out how the size and diversity of the datasets affect deep models' performance. Finally, a systematic survey to the existing efforts to solve the data hunger problem is conducted on both methodological and dataset's viewpoints, followed by an insightful discussion of remaining problems and open questions To the best of our knowledge, this is the first work to analyze the data hunger problem for 3D semantic segmentation using deep learning techniques that are addressed in the literature review, statistical analysis, and cross-dataset and cross-algorithm experiments. We share findings and discussions, which may lead to potential topics in future works.
arXiv comment22 pages, 15 figures
References in corpus (43)
- Deep Learning in Neural Networks: An Overview
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Prototypical Networks for Few-shot Learning
- Learning Transferable Features with Deep Adaptation Networks
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
- Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
- A Review on Deep Learning Techniques Applied to Semantic Segmentation
- The ApolloScape Open Dataset for Autonomous Driving and its Application
- MixMatch: A Holistic Approach to Semi-Supervised Learning
- ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
- Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation
- Deep Learning: A Critical Appraisal
- PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation
- CARLA: An Open Urban Driving Simulator
- Vehicle Detection from 3D Lidar Using Fully Convolutional Network
- Fully Convolutional Multi-Class Multiple Instance Learning
- A2D2: Audi Autonomous Driving Dataset
- Scalability in Perception for Autonomous Driving: Waymo Open Dataset
- Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation
- Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networks
- Augmented LiDAR Simulator for Autonomous Driving
- Argoverse: 3D Tracking and Forecasting with Rich Maps
- PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud
- SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving
- Colorization as a Proxy Task for Visual Understanding
- SynthCity: A large scale synthetic point cloud
- Deep Learning for 3D Point Clouds: A Survey
- A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving
- Weakly Supervised Semantic Point Cloud Segmentation:Towards 10X Fewer Labels
- CEREALS - Cost-Effective REgion-based Active Learning for Semantic Segmentation
- LSANet: Feature Learning on Point Sets by Local Spatial Aware Layer
- RIU-Net: Embarrassingly simple semantic segmentation of 3D LiDAR point cloud
- Learning a Spatio-Temporal Embedding for Video Instance Segmentation
- SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances
- DAR-Net: Dynamic Aggregation Network for Semantic Scene Segmentation
- Improving 3D Object Detection through Progressive Population Based Augmentation
- VolMap: A Real-time Model for Semantic Segmentation of a LiDAR surrounding view
- Zero-Shot Semantic Segmentation
- PointAtrousGraph: Deep Hierarchical Encoder-Decoder with Point Atrous Convolution for Unorganized 3D Points
- Differentiable Meta-learning Model for Few-shot Semantic Segmentation
- Review: deep learning on 3D point clouds
- End-to-End 3D-PointCloud Semantic Segmentation for Autonomous Driving