An Empirical Evaluation of Deep Learning on Highway Driving

Abstract

Numerous groups have applied a variety of deep learning techniques to computer vision problems in highway perception scenarios. In this paper, we presented a number of empirical evaluations of recent deep learning advances. Computer vision, combined with deep learning, has the potential to bring about a relatively inexpensive, robust solution to autonomous driving. To prepare deep learning for industry uptake and practical applications, neural networks will require large data sets that represent all possible driving environments and scenarios. We collect a large data set of highway data and apply deep learning and computer vision algorithms to problems such as car and lane detection. We show how existing convolutional neural networks (CNNs) can be used to perform lane and vehicle detection while running at frame rates required for a real-time system. Our results lend credence to the hypothesis that deep learning holds promise for autonomous driving.

arXiv commentAdded a video for lane detection

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Going Deeper with Convolutions
• Scalable, High-Quality Object Detection

Cited by in corpus (81)

• A Survey of Autonomous Driving: Common Practices and Emerging Technologies
• Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• Deep Machine Learning Approach to Develop a New Asphalt Pavement Condition Index
• CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices
• Deep Learning in the Automotive Industry: Applications and Tools
• Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
• End-to-End Deep Reinforcement Learning for Lane Keeping Assist
• TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
• Gen-LaneNet: A Generalized and Scalable Approach for 3D Lane Detection
• Characterizing Driving Styles with Deep Learning
• LaneNet: Real-Time Lane Detection Networks for Autonomous Driving
• Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars
• TBD: Benchmarking and Analyzing Deep Neural Network Training
• Driving Scene Perception Network: Real-time Joint Detection, Depth Estimation and Semantic Segmentation
• Characterizing the Decision Boundary of Deep Neural Networks
• Deep Reinforcement Learning for Autonomous Driving
• End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perception
• Shedding Light on Blind Spots: Developing a Reference Architecture to Leverage Video Data for Process Mining
• End-to-end Learning of Driving Models from Large-scale Video Datasets
• LDNet: End-to-End Lane Marking Detection Approach Using a Dynamic Vision Sensor
• On the Robustness of Convolutional Neural Networks to Internal Architecture and Weight Perturbations
• DeepCruiser: Automated Guided Testing for Stateful Deep Learning Systems
• Learning from Maps: Visual Common Sense for Autonomous Driving
• 3D-LaneNet+: Anchor Free Lane Detection using a Semi-Local Representation
• End-to-end people detection in crowded scenes
• RESA: Recurrent Feature-Shift Aggregator for Lane Detection
• End to End Video Segmentation for Driving : Lane Detection For Autonomous Car
• Autonomous Braking System via Deep Reinforcement Learning
• Software Engineering Practice in the Development of Deep Learning Applications
• Structure-Aware Network for Lane Marker Extraction with Dynamic Vision Sensor
• Improving Interpretability of Deep Neural Networks with Semantic Information
• Uniform Error Bounds for Gaussian Process Regression with Application to Safe Control
• RoboNet: Large-Scale Multi-Robot Learning
• Coverage Testing of Deep Learning Models using Dataset Characterization
• Ultra Fast Structure-aware Deep Lane Detection
• Can Single Neurons Solve MNIST? The Computational Power of Biological Dendritic Trees
• Uniform Error and Posterior Variance Bounds for Gaussian Process Regression with Application to Safe Control
• Interpreting and Evaluating Neural Network Robustness
• Brain Inspired Cognitive Model with Attention for Self-Driving Cars
• Incorporating Unlabeled Data into Distributionally Robust Learning
• Structured Weight Matrices-Based Hardware Accelerators in Deep Neural Networks: FPGAs and ASICs
• Driver Behavior Analysis Using Lane Departure Detection Under Challenging Conditions
• A Fog Computing Framework for Autonomous Driving Assist: Architecture, Experiments, and Challenges
• Deep Object-Centric Policies for Autonomous Driving
• Relative stability toward diffeomorphisms indicates performance in deep nets
• Deep Learning and Control Algorithms of Direct Perception for Autonomous Driving
• Verification of Binarized Neural Networks via Inter-Neuron Factoring
• Semi-Local 3D Lane Detection and Uncertainty Estimation
• Driverseat: Crowdstrapping Learning Tasks for Autonomous Driving
• FFT-Based Deep Learning Deployment in Embedded Systems
• Tuned Inception V3 for Recognizing States of Cooking Ingredients
• SEALing Neural Network Models in Secure Deep Learning Accelerators
• Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training
• An Empirical Study of Explainable AI Techniques on Deep Learning Models For Time Series Tasks
• Driving Experience Transfer Method for End-to-End Control of Self-Driving Cars
• MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving
• WAD: A Deep Reinforcement Learning Agent for Urban Autonomous Driving
• 3D-LaneNet: End-to-End 3D Multiple Lane Detection
• Analyzing Accuracy Loss in Randomized Smoothing Defenses
• Multi-modal Experts Network for Autonomous Driving
• An Evolution of CNN Object Classifiers on Low-Resolution Images
• Deep Mixture Density Network for Probabilistic Object Detection
• Quantitative Projection Coverage for Testing ML-enabled Autonomous Systems
• nn-dependability-kit: Engineering Neural Networks for Safety-Critical Autonomous Driving Systems
• Fractional moment-preserving initialization schemes for training deep neural networks
• Classifying neuromorphic data using a deep learning framework for image classification
• Instance-Aware Predictive Navigation in Multi-Agent Environments
• Vehicle Detection in Deep Learning
• Defending Against Adversarial Denial-of-Service Data Poisoning Attacks
• Real-Time Road Segmentation Using LiDAR Data Processing on an FPGA
• On the Adversarial Robustness of Multivariate Robust Estimation
• PipeTune: Pipeline Parallelism of Hyper and System Parameters Tuning for Deep Learning Clusters
• Multi-lane Detection Using Instance Segmentation and Attentive Voting
• A Dataset for Lane Instance Segmentation in Urban Environments
• Network Horizon Dynamics I: Qualitative Aspects
• The Aleatoric Uncertainty Estimation Using a Separate Formulation with Virtual Residuals
• Distributionally Robust Learning with Stable Adversarial Training
• Deep Multi-Sensor Lane Detection
• Automatic Vector-based Road Structure Mapping Using Multi-beam LiDAR
• Hierarchical Recurrent Attention Networks for Structured Online Maps