Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling

Abstract

We study the problem of efficient semantic segmentation of large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net, an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Comparative experiments show that our RandLA-Net can process 1 million points in a single pass up to 200x faster than existing approaches. Moreover, extensive experiments on five large-scale point cloud datasets, including Semantic3D, SemanticKITTI, Toronto3D, NPM3D and S3DIS, demonstrate the state-of-the-art semantic segmentation performance of our RandLA-Net.

arXiv commentIEEE TPAMI 2021. arXiv admin note: substantial text overlap with arXiv:1911.11236

References in corpus (11)

• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• Deep Parametric Continuous Convolutional Neural Networks
• Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways
• Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds
• Neural Variational Inference and Learning in Belief Networks
• Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction
• DeepGCNs: Can GCNs Go as Deep as CNNs?
• ShellNet: Efficient Point Cloud Convolutional Neural Networks using Concentric Shells Statistics
• Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs
• PCAN: 3D Attention Map Learning Using Contextual Information for Point Cloud Based Retrieval
• Deformable Filter Convolution for Point Cloud Reasoning

Cited by in corpus (5)

• Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving
• Small but Mighty: Enhancing 3D Point Clouds Semantic Segmentation with U-Next Framework
• Spatial-Temporal Transformer for 3D Point Cloud Sequences
• Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation
• LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Driving