A Systematic Comparison of Deep Learning Architectures in an Autonomous Vehicle

Abstract

Self-driving technology is advancing rapidly --- albeit with significant challenges and limitations. This progress is largely due to recent developments in deep learning algorithms. To date, however, there has been no systematic comparison of how different deep learning architectures perform at such tasks, or an attempt to determine a correlation between classification performance and performance in an actual vehicle, a potentially critical factor in developing self-driving systems. Here, we introduce the first controlled comparison of multiple deep-learning architectures in an end-to-end autonomous driving task across multiple testing conditions. We compared performance, under identical driving conditions, across seven architectures including a fully-connected network, a simple 2 layer CNN, AlexNet, VGG-16, Inception-V3, ResNet, and an LSTM by assessing the number of laps each model was able to successfully complete without crashing while traversing an indoor racetrack. We compared performance across models when the conditions exactly matched those in training as well as when the local environment and track were configured differently and objects that were not included in the training dataset were placed on the track in various positions. In addition, we considered performance using several different data types for training and testing including single grayscale and color frames, and multiple grayscale frames stacked together in sequence. With the exception of a fully-connected network, all models performed reasonably well (around or above 80\%) and most very well (~95\%) on at least one input type but with considerable variation across models and inputs. Overall, AlexNet, operating on single color frames as input, achieved the best level of performance (100\% success rate in phase one and 55\% in phase two) while VGG-16 performed well most consistently across image types.

arXiv comment16 pages, 14 figures, 2 tables

References in corpus (15)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Scikit-learn: Machine Learning in Python
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• End to End Learning for Self-Driving Cars
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Fully Convolutional Networks for Semantic Segmentation
• Learning to Segment Object Candidates
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
• L2 Regularization versus Batch and Weight Normalization
• Addressing the Rare Word Problem in Neural Machine Translation
• Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks
• CityPersons: A Diverse Dataset for Pedestrian Detection
• Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars
• End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perception