AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithms

Abstract

Simulation systems have become an essential component in the development and validation of autonomous driving technologies. The prevailing state-of-the-art approach for simulation is to use game engines or high-fidelity computer graphics (CG) models to create driving scenarios. However, creating CG models and vehicle movements (e.g., the assets for simulation) remains a manual task that can be costly and time-consuming. In addition, the fidelity of CG images still lacks the richness and authenticity of real-world images and using these images for training leads to degraded performance. In this paper we present a novel approach to address these issues: Augmented Autonomous Driving Simulation (AADS). Our formulation augments real-world pictures with a simulated traffic flow to create photo-realistic simulation images and renderings. More specifically, we use LiDAR and cameras to scan street scenes. From the acquired trajectory data, we generate highly plausible traffic flows for cars and pedestrians and compose them into the background. The composite images can be re-synthesized with different viewpoints and sensor models. The resulting images are photo-realistic, fully annotated, and ready for end-to-end training and testing of autonomous driving systems from perception to planning. We explain our system design and validate our algorithms with a number of autonomous driving tasks from detection to segmentation and predictions. Compared to traditional approaches, our method offers unmatched scalability and realism. Scalability is particularly important for AD simulation and we believe the complexity and diversity of the real world cannot be realistically captured in a virtual environment. Our augmented approach combines the flexibility in a virtual environment (e.g., vehicle movements) with the richness of the real world to allow effective simulation of anywhere on earth.

References in corpus (9)

• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• CARLA: An Open Urban Driving Simulator
• Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications
• Augmented LiDAR Simulator for Autonomous Driving
• SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting
• A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving
• AutonoVi: Autonomous Vehicle Planning with Dynamic Maneuvers and Traffic Constraints
• Heter-Sim: Heterogeneous multi-agent systems simulation by interactive data-driven optimization

Cited by in corpus (17)

• MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving
• A Study on the Challenges of Using Robotics Simulators for Testing
• Modeling Lead-vehicle Kinematics For Rear-end Crash Scenario Generation
• All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles
• Neural Network Generalization: The impact of camera parameters
• PerMO: Perceiving More at Once from a Single Image for Autonomous Driving
• SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Driving
• GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving
• Breaking the Limits of Remote Sensing by Simulation and Deep Learning for Flood and Debris Flow Mapping
• Vehicle Reconstruction and Texture Estimation Using Deep Implicit Semantic Template Mapping
• Spatio-temporal Keyframe Control of Traffic Simulation using Coarse-to-Fine Optimization
• A Learning-based Stochastic Driving Model for Autonomous Vehicle Testing
• DVI: Depth Guided Video Inpainting for Autonomous Driving
• AutoRemover: Automatic Object Removal for Autonomous Driving Videos
• Synthetic training data generation for deep learning based quality inspection
• KIT Bus: A Shuttle Model for CARLA Simulator
• Enhanced Transfer Learning for Autonomous Driving with Systematic Accident Simulation