There is Limited Correlation between Coverage and Robustness for Deep Neural Networks
arXiv:1911.05904
Abstract
Deep neural networks (DNN) are increasingly applied in safety-critical systems, e.g., for face recognition, autonomous car control and malware detection. It is also shown that DNNs are subject to attacks such as adversarial perturbation and thus must be properly tested. Many coverage criteria for DNN since have been proposed, inspired by the success of code coverage criteria for software programs. The expectation is that if a DNN is a well tested (and retrained) according to such coverage criteria, it is more likely to be robust. In this work, we conduct an empirical study to evaluate the relationship between coverage, robustness and attack/defense metrics for DNN. Our study is the largest to date and systematically done based on 100 DNN models and 25 metrics. One of our findings is that there is limited correlation between coverage and robustness, i.e., improving coverage does not help improve the robustness. Our dataset and implementation have been made available to serve as a benchmark for future studies on testing DNN.
References in corpus (18)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- TensorFlow: A system for large-scale machine learning
- End to End Learning for Self-Driving Cars
- DeepXplore: Automated Whitebox Testing of Deep Learning Systems
- Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
- Adversarial Machine Learning at Scale
- Robustness May Be at Odds with Accuracy
- Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
- Adversarial Examples: Opportunities and Challenges
- Testing Deep Neural Networks
- Parseval Networks: Improving Robustness to Adversarial Examples
- Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
- Lipschitz regularity of deep neural networks: analysis and efficient estimation
- TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
- Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks
- Concolic Testing for Deep Neural Networks
- DeepMutation: Mutation Testing of Deep Learning Systems
- Towards Imperceptible and Robust Adversarial Example Attacks against Neural Networks