Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars

Abstract

As an initial assessment, over 480,000 labeled virtual images of normal highway driving were readily generated in Grand Theft Auto V's virtual environment. Using these images, a CNN was trained to detect following distance to cars/objects ahead, lane markings, and driving angle (angular heading relative to lane centerline): all variables necessary for basic autonomous driving. Encouraging results were obtained when tested on over 50,000 labeled virtual images from substantially different GTA-V driving environments. This initial assessment begins to define both the range and scope of the labeled images needed for training as well as the range and scope of labeled images needed for testing the definition of boundaries and limitations of trained networks. It is the efficacy and flexibility of a "GTA-V"-like virtual environment that is expected to provide an efficient well-defined foundation for the training and testing of Convolutional Neural Networks for safe driving. Additionally, described is the Princeton Virtual Environment (PVE) for the training, testing and enhancement of safe driving AI, which is being developed using the video-game engine Unity. PVE is being developed to recreate rare but critical corner cases that can be used in re-training and enhancing machine learning models and understanding the limitations of current self driving models. The Florida Tesla crash is being used as an initial reference.

15 pages, 4 figures, under review by TRB 2018 Annual Meeting

References in corpus (3)

• An Empirical Evaluation of Deep Learning on Highway Driving
• DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
• End-to-end Learning of Driving Models from Large-scale Video Datasets

Cited by in corpus (12)

• Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving
• FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusion
• BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloud
• Predicting drag on rough surfaces by transfer learning of empirical correlations
• Improving the Generalization of End-to-End Driving through Procedural Generation
• SynthText3D: Synthesizing Scene Text Images from 3D Virtual Worlds
• GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors
• A Systematic Comparison of Deep Learning Architectures in an Autonomous Vehicle
• Single View Metrology in the Wild
• A gamified simulator and physical platform for self-driving algorithm training and validation
• Enhanced Transfer Learning for Autonomous Driving with Systematic Accident Simulation