Learning a Driving Simulator

Abstract

Comma.ai's approach to Artificial Intelligence for self-driving cars is based on an agent that learns to clone driver behaviors and plans maneuvers by simulating future events in the road. This paper illustrates one of our research approaches for driving simulation. One where we learn to simulate. Here we investigate variational autoencoders with classical and learned cost functions using generative adversarial networks for embedding road frames. Afterwards, we learn a transition model in the embedded space using action conditioned Recurrent Neural Networks. We show that our approach can keep predicting realistic looking video for several frames despite the transition model being optimized without a cost function in the pixel space.

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Adversarial Feature Learning
• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Deep multi-scale video prediction beyond mean square error
• Discriminative Regularization for Generative Models
• On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models

Cited by in corpus (44)

• A Survey of Autonomous Driving: Common Practices and Emerging Technologies
• Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives
• Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
• Multimodal End-to-End Autonomous Driving
• A Review on Deep Learning Techniques for Video Prediction
• A Survey of End-to-End Driving: Architectures and Training Methods
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Recurrent Neural Networks: An Embedded Computing Perspective
• Self-Driving Car Steering Angle Prediction Based on Image Recognition
• End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies
• A Commute in Data: The comma2k19 Dataset
• Experimental Resilience Assessment of An Open-Source Driving Agent
• End-to-end Learning of Driving Models from Large-scale Video Datasets
• Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
• Geo-Context Aware Study of Vision-Based Autonomous Driving Models and Spatial Video Data
• Self-driving scale car trained by Deep reinforcement learning
• Driving Datasets Literature Review
• Is it Safe to Drive? An Overview of Factors, Challenges, and Datasets for Driveability Assessment in Autonomous Driving
• Transfer learning from synthetic to real images using variational autoencoders for robotic applications
• A Little Fog for a Large Turn
• Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks
• Two-stream convolutional networks for end-to-end learning of self-driving cars
• On the Road with 16 Neurons: Mental Imagery with Bio-inspired Deep Neural Networks
• DeepRacing: Parameterized Trajectories for Autonomous Racing
• SAD-GAN: Synthetic Autonomous Driving using Generative Adversarial Networks
• SAINT-ACC: Safety-Aware Intelligent Adaptive Cruise Control for Autonomous Vehicles Using Deep Reinforcement Learning
• Visuomotor Understanding for Representation Learning of Driving Scenes
• Multi-task learning with deep model based reinforcement learning
• Improving Robustness of Learning-based Autonomous Steering Using Adversarial Images
• Semi-supervised Learning: Fusion of Self-supervised, Supervised Learning, and Multimodal Cues for Tactical Driver Behavior Detection
• Fonts-2-Handwriting: A Seed-Augment-Train framework for universal digit classification
• Behavioral Research and Practical Models of Drivers' Attention
• Deep-n-Cheap: An Automated Search Framework for Low Complexity Deep Learning
• A Methodological Review of Visual Road Recognition Procedures for Autonomous Driving Applications
• CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving
• DDD20 End-to-End Event Camera Driving Dataset: Fusing Frames and Events with Deep Learning for Improved Steering Prediction
• D-ACC: Dynamic Adaptive Cruise Control for Highways with Ramps Based on Deep Q-Learning
• Brain-Inspired Deep Imitation Learning for Autonomous Driving Systems
• Tackling Variabilities in Autonomous Driving
• Video Compression Coding via Colorization: A Generative Adversarial Network (GAN)-Based Approach
• Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty
• On Offline Evaluation of Vision-based Driving Models
• Traversing Latent Space using Decision Ferns
• CAN Coach: Vehicular Control through Human Cyber-Physical Systems