DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

Abstract

Recent advances in Deep Neural Networks (DNNs) have led to the development of DNN-driven autonomous cars that, using sensors like camera, LiDAR, etc., can drive without any human intervention. Most major manufacturers including Tesla, GM, Ford, BMW, and Waymo/Google are working on building and testing different types of autonomous vehicles. The lawmakers of several US states including California, Texas, and New York have passed new legislation to fast-track the process of testing and deployment of autonomous vehicles on their roads. However, despite their spectacular progress, DNNs, just like traditional software, often demonstrate incorrect or unexpected corner case behaviors that can lead to potentially fatal collisions. Several such real-world accidents involving autonomous cars have already happened including one which resulted in a fatality. Most existing testing techniques for DNN-driven vehicles are heavily dependent on the manual collection of test data under different driving conditions which become prohibitively expensive as the number of test conditions increases. In this paper, we design, implement and evaluate DeepTest, a systematic testing tool for automatically detecting erroneous behaviors of DNN-driven vehicles that can potentially lead to fatal crashes. First, our tool is designed to automatically generated test cases leveraging real-world changes in driving conditions like rain, fog, lighting conditions, etc. DeepTest systematically explores different parts of the DNN logic by generating test inputs that maximize the numbers of activated neurons. DeepTest found thousands of erroneous behaviors under different realistic driving conditions (e.g., blurring, rain, fog, etc.) many of which lead to potentially fatal crashes in three top performing DNNs in the Udacity self-driving car challenge.

References in corpus (10)

• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Robust Physical-World Attacks on Deep Learning Models
• Parseval Networks: Improving Robustness to Adversarial Examples
• Simple Black-Box Adversarial Perturbations for Deep Networks
• Extending Defensive Distillation
• Certified Defenses for Data Poisoning Attacks
• Adversarial examples for generative models

Cited by in corpus (33)

• DLFuzz: Differential Fuzzing Testing of Deep Learning Systems
• Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks
• Automated Directed Fairness Testing
• Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testing
• Testing Deep Neural Networks
• TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
• Algorithmic decision-making in AVs: Understanding ethical and technical concerns for smart cities
• Verification for Machine Learning, Autonomy, and Neural Networks Survey
• DriveFuzz: Discovering Autonomous Driving Bugs through Driving Quality-Guided Fuzzing
• Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems
• DeepRoad: GAN-based Metamorphic Autonomous Driving System Testing
• Ensemble Neural Networks (ENN): A gradient-free stochastic method
• Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspective
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving
• Experimental Resilience Assessment of An Open-Source Driving Agent
• Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing
• On the Replicability and Reproducibility of Deep Learning in Software Engineering
• Understanding Performance Problems in Deep Learning Systems
• Software Testing for Machine Learning
• Deep Learning in Software Engineering
• Global Robustness Evaluation of Deep Neural Networks with Provable Guarantees for the $L_0$ Norm
• Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer
• Road Context-aware Intrusion Detection System for Autonomous Cars
• A Little Fog for a Large Turn
• M-to-N Backdoor Paradigm: A Multi-Trigger and Multi-Target Attack to Deep Learning Models
• EREBA: Black-box Energy Testing of Adaptive Neural Networks
• Revisiting Deep Neural Network Test Coverage from the Test Effectiveness Perspective
• Automated Testing for Deep Learning Systems with Differential Behavior Criteria
• Manifestation of Image Contrast in Deep Networks
• Metamorphic Testing of a Deep Learning based Forecaster
• DeepGuard: A Framework for Safeguarding Autonomous Driving Systems from Inconsistent Behavior
• Formal Verification of Neural Network Controlled Autonomous Systems