PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud

Abstract

In this paper, we propose PointSeg, a real-time end-to-end semantic segmentation method for road-objects based on spherical images. We take the spherical image, which is transformed from the 3D LiDAR point clouds, as input of the convolutional neural networks (CNNs) to predict the point-wise semantic map. To make PointSeg applicable on a mobile system, we build the model based on the light-weight network, SqueezeNet, with several improvements. It maintains a good balance between memory cost and prediction performance. Our model is trained on spherical images and label masks projected from the KITTI 3D object detection dataset. Experiments show that PointSeg can achieve competitive accuracy with 90fps on a single GPU 1080ti. which makes it quite compatible for autonomous driving applications.

arXiv commentVideo link: https://youtu.be/b1BAbcjJ10s code link: https://github.com/ywangeq/PointSeg

References in corpus (8)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Rethinking Atrous Convolution for Semantic Image Segmentation
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Multi-Scale Context Aggregation by Dilated Convolutions
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
• ICNet for Real-Time Semantic Segmentation on High-Resolution Images
• Understanding Convolution for Semantic Segmentation

Cited by in corpus (22)

• SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving
• Towards Robust LiDAR-based Perception in Autonomous Driving: General Black-box Adversarial Sensor Attack and Countermeasures
• L2G Auto-encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention
• FPS-Net: A Convolutional Fusion Network for Large-Scale LiDAR Point Cloud Segmentation
• RangeSeg: Range-Aware Real Time Segmentation of 3D LiDAR Point Clouds
• Deep Learning Based 3D Segmentation: A Survey
• SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud
• RGB and LiDAR fusion based 3D Semantic Segmentation for Autonomous Driving
• CAE-LO: LiDAR Odometry Leveraging Fully Unsupervised Convolutional Auto-Encoder for Interest Point Detection and Feature Description
• RIU-Net: Embarrassingly simple semantic segmentation of 3D LiDAR point cloud
• SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances
• PASS3D: Precise and Accelerated Semantic Segmentation for 3D Point Cloud
• Efficient Deep Neural Networks
• VolMap: A Real-time Model for Semantic Segmentation of a LiDAR surrounding view
• S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Clouds
• PointIT: A Fast Tracking Framework Based on 3D Instance Segmentation
• 3D-DEEP: 3-Dimensional Deep-learning based on elevation patterns forroad scene interpretation
• Weighted Bayesian Gaussian Mixture Model for Roadside LiDAR Object Detection
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
• Joint Pose and Shape Estimation of Vehicles from LiDAR Data
• Human Segmentation with Dynamic LiDAR Data
• LO-Net: Deep Real-time Lidar Odometry