A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving

Abstract

3D LiDAR scanners are playing an increasingly important role in autonomous driving as they can generate depth information of the environment. However, creating large 3D LiDAR point cloud datasets with point-level labels requires a significant amount of manual annotation. This jeopardizes the efficient development of supervised deep learning algorithms which are often data-hungry. We present a framework to rapidly create point clouds with accurate point-level labels from a computer game. The framework supports data collection from both auto-driving scenes and user-configured scenes. Point clouds from auto-driving scenes can be used as training data for deep learning algorithms, while point clouds from user-configured scenes can be used to systematically test the vulnerability of a neural network, and use the falsifying examples to make the neural network more robust through retraining. In addition, the scene images can be captured simultaneously in order for sensor fusion tasks, with a method proposed to do automatic calibration between the point clouds and captured scene images. We show a significant improvement in accuracy (+9%) in point cloud segmentation by augmenting the training dataset with the generated synthesized data. Our experiments also show by testing and retraining the network using point clouds from user-configured scenes, the weakness/blind spots of the neural network can be fixed.

References in corpus (5)

• CARLA: An Open Urban Driving Simulator
• Playing for Data: Ground Truth from Computer Games
• Systematic Testing of Convolutional Neural Networks for Autonomous Driving
• Detecting Drivable Area for Self-driving Cars: An Unsupervised Approach
• SmartAnnotator: An Interactive Tool for Annotating RGBD Indoor Images

Cited by in corpus (11)

• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
• AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithms
• Augmented LiDAR Simulator for Autonomous Driving
• Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Level
• PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention
• Task-Aware Sampling Layer for Point-Wise Analysis
• ePointDA: An End-to-End Simulation-to-Real Domain Adaptation Framework for LiDAR Point Cloud Segmentation
• KTNet: Knowledge Transfer for Unpaired 3D Shape Completion
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
• FLAVA: Find, Localize, Adjust and Verify to Annotate LiDAR-Based Point Clouds
• GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D LiDAR Segmentation