End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies

Abstract

Steering a car through traffic is a complex task that is difficult to cast into algorithms. Therefore, researchers turn to training artificial neural networks from front-facing camera data stream along with the associated steering angles. Nevertheless, most existing solutions consider only the visual camera frames as input, thus ignoring the temporal relationship between frames. In this work, we propose a Convolutional Long Short-Term Memory Recurrent Neural Network (C-LSTM), that is end-to-end trainable, to learn both visual and dynamic temporal dependencies of driving. Additionally, We introduce posing the steering angle regression problem as classification while imposing a spatial relationship between the output layer neurons. Such method is based on learning a sinusoidal function that encodes steering angles. To train and validate our proposed methods, we used the publicly available Comma.ai dataset. Our solution improved steering root mean square error by 35% over recent methods, and led to a more stable steering by 87%.

31st Conference on Neural Information Processing Systems (NIPS), Machine Learning for Intelligent Transportation Systems Workshop, Long Beach, CA, USA, 2017

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• End to End Learning for Self-Driving Cars
• Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
• Learning a Driving Simulator
• End-to-end Learning of Driving Models from Large-scale Video Datasets
• Reactive Collision Avoidance using Evolutionary Neural Networks

Cited by in corpus (17)

• A Survey of Deep Learning Techniques for Autonomous Driving
• Multimodal End-to-End Autonomous Driving
• A Survey of Deep RL and IL for Autonomous Driving Policy Learning
• F1/10: An Open-Source Autonomous Cyber-Physical Platform
• Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks
• Social Attention for Autonomous Decision-Making in Dense Traffic
• Learning Loss for Test-Time Augmentation
• A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning
• Two-stream convolutional networks for end-to-end learning of self-driving cars
• End-to-end driving simulation via angle branched network
• Improving Generalization of Transfer Learning Across Domains Using Spatio-Temporal Features in Autonomous Driving
• On the Road with 16 Neurons: Mental Imagery with Bio-inspired Deep Neural Networks
• DeepRacing: Parameterized Trajectories for Autonomous Racing
• Real-time End-to-End Federated Learning: An Automotive Case Study
• Vision-based Navigation of Autonomous Vehicle in Roadway Environments with Unexpected Hazards
• Foresee: Attentive Future Projections of Chaotic Road Environments with Online Training
• Driver Distraction Identification with an Ensemble of Convolutional Neural Networks