Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing

Abstract

Recently, it has been shown that deep neural networks (DNN) are subject to attacks through adversarial samples. Adversarial samples are often crafted through adversarial perturbation, i.e., manipulating the original sample with minor modifications so that the DNN model labels the sample incorrectly. Given that it is almost impossible to train perfect DNN, adversarial samples are shown to be easy to generate. As DNN are increasingly used in safety-critical systems like autonomous cars, it is crucial to develop techniques for defending such attacks. Existing defense mechanisms which aim to make adversarial perturbation challenging have been shown to be ineffective. In this work, we propose an alternative approach. We first observe that adversarial samples are much more sensitive to perturbations than normal samples. That is, if we impose random perturbations on a normal and an adversarial sample respectively, there is a significant difference between the ratio of label change due to the perturbations. Observing this, we design a statistical adversary detection algorithm called nMutant (inspired by mutation testing from software engineering community). Our experiments show that nMutant effectively detects most of the adversarial samples generated by recently proposed attacking methods. Furthermore, we provide an error bound with certain statistical significance along with the detection.

arXiv commentSumitted to NIPS 2018

References in corpus (22)

• Ensemble Adversarial Training: Attacks and Defenses
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Countering Adversarial Images using Input Transformations
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• On the (Statistical) Detection of Adversarial Examples
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Spatially Transformed Adversarial Examples
• DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars
• Parseval Networks: Improving Robustness to Adversarial Examples
• Towards Proving the Adversarial Robustness of Deep Neural Networks
• Fast Feature Fool: A data independent approach to universal adversarial perturbations
• Extending Defensive Distillation
• EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
• Attacking Binarized Neural Networks
• Attacking Convolutional Neural Network using Differential Evolution

Cited by in corpus (15)

• Testing Deep Neural Networks
• Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing
• Machine Learning Testing: Survey, Landscapes and Horizons
• Software Engineering for AI-Based Systems: A Survey
• Toward Robust Image Classification
• Towards Security Threats of Deep Learning Systems: A Survey
• Input Validation for Neural Networks via Runtime Local Robustness Verification
• Efficient Two-Step Adversarial Defense for Deep Neural Networks
• Automatic Fairness Testing of Neural Classifiers through Adversarial Sampling
• Reliability Validation of Learning Enabled Vehicle Tracking
• Explaining Image Classifiers using Statistical Fault Localization
• Efficient detection of adversarial images
• Beating Attackers At Their Own Games: Adversarial Example Detection Using Adversarial Gradient Directions
• Boosting the Robustness Verification of DNN by Identifying the Achilles's Heel
• Leveraging Model Interpretability and Stability to increase Model Robustness