SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving

Abstract

In this paper, we introduce SalsaNext for the uncertainty-aware semantic segmentation of a full 3D LiDAR point cloud in real-time. SalsaNext is the next version of SalsaNet [1] which has an encoder-decoder architecture where the encoder unit has a set of ResNet blocks and the decoder part combines upsampled features from the residual blocks. In contrast to SalsaNet, we introduce a new context module, replace the ResNet encoder blocks with a new residual dilated convolution stack with gradually increasing receptive fields and add the pixel-shuffle layer in the decoder. Additionally, we switch from stride convolution to average pooling and also apply central dropout treatment. To directly optimize the Jaccard index, we further combine the weighted cross-entropy loss with Lovasz-Softmax loss [2]. We finally inject a Bayesian treatment to compute the epistemic and aleatoric uncertainties for each point in the cloud. We provide a thorough quantitative evaluation on the Semantic-KITTI dataset [3], which demonstrates that the proposed SalsaNext outperforms other state-of-the-art semantic segmentation networks and ranks first on the Semantic-KITTI leaderboard. We also release our source code https://github.com/TiagoCortinhal/SalsaNext.

References in corpus (7)

• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Visualizing and Understanding Convolutional Networks
• Fast-SCNN: Fast Semantic Segmentation Network
• Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding
• PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud
• SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Driving
• Point Cloud Instance Segmentation using Probabilistic Embeddings

Cited by in corpus (28)

• A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving
• Dynamic Object Aware LiDAR SLAM based on Automatic Generation of Training Data
• FPS-Net: A Convolutional Fusion Network for Large-Scale LiDAR Point Cloud Segmentation
• KPRNet: Improving projection-based LiDAR semantic segmentation
• Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
• Deep Learning on 3D Semantic Segmentation: A Detailed Review
• Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges
• RELLIS-3D Dataset: Data, Benchmarks and Analysis
• AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentation
• TORNADO-Net: mulTiview tOtal vaRiatioN semAntic segmentation with Diamond inceptiOn module
• LiDARNet: A Boundary-Aware Domain Adaptation Model for Point Cloud Semantic Segmentation
• Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation
• PolarStream: Streaming Lidar Object Detection and Segmentation with Polar Pillars
• S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Clouds
• SemCal: Semantic LiDAR-Camera Calibration using Neural MutualInformation Estimator
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
• VIN: Voxel-based Implicit Network for Joint 3D Object Detection and Segmentation for Lidars
• NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes
• SA-LOAM: Semantic-aided LiDAR SLAM with Loop Closure
• HPGNN: Using Hierarchical Graph Neural Networks for Outdoor Point Cloud Processing
• Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning
• Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation
• Efficient and Robust LiDAR-Based End-to-End Navigation
• S3Net: 3D LiDAR Sparse Semantic Segmentation Network
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception
• Point Cloud Segmentation Using Sparse Temporal Local Attention
• LiDAR-based Recurrent 3D Semantic Segmentation with Temporal Memory Alignment