Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing

Abstract

Deep neural networks (DNN) have been shown to be useful in a wide range of applications. However, they are also known to be vulnerable to adversarial samples. By transforming a normal sample with some carefully crafted human imperceptible perturbations, even highly accurate DNN make wrong decisions. Multiple defense mechanisms have been proposed which aim to hinder the generation of such adversarial samples. However, a recent work show that most of them are ineffective. In this work, we propose an alternative approach to detect adversarial samples at runtime. Our main observation is that adversarial samples are much more sensitive than normal samples if we impose random mutations on the DNN. We thus first propose a measure of `sensitivity' and show empirically that normal samples and adversarial samples have distinguishable sensitivity. We then integrate statistical hypothesis testing and model mutation testing to check whether an input sample is likely to be normal or adversarial at runtime by measuring its sensitivity. We evaluated our approach on the MNIST and CIFAR10 datasets. The results show that our approach detects adversarial samples generated by state-of-the-art attacking methods efficiently and accurately.

arXiv commentAccepted by ICSE 2019

References in corpus (18)

• Intriguing properties of neural networks
• Ensemble Adversarial Training: Attacks and Defenses
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Achieving Human Parity in Conversational Speech Recognition
• Countering Adversarial Images using Input Transformations
• Detecting Adversarial Samples from Artifacts
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Spatially Transformed Adversarial Examples
• Mitigating Adversarial Effects Through Randomization
• Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
• Towards Fast Computation of Certified Robustness for ReLU Networks
• Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models
• Towards Proving the Adversarial Robustness of Deep Neural Networks
• Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing
• Concolic Testing for Deep Neural Networks
• Attacking Binarized Neural Networks
• Towards Robust Training of Neural Networks by Regularizing Adversarial Gradients

Cited by in corpus (12)

• Dynamic Slicing for Deep Neural Networks
• Testing DNN Image Classifiers for Confusion & Bias Errors
• COMET: Coverage-guided Model Generation For Deep Learning Library Testing
• Increasing the Confidence of Deep Neural Networks by Coverage Analysis
• Operational Calibration: Debugging Confidence Errors for DNNs in the Field
• Code Integrity Attestation for PLCs using Black Box Neural Network Predictions
• PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing
• Determining Sequence of Image Processing Technique (IPT) to Detect Adversarial Attacks
• QuanTest: Entanglement-Guided Testing of Quantum Neural Network Systems
• TEASMA: A Practical Methodology for Test Adequacy Assessment of Deep Neural Networks
• Testing Deep Learning Models: A First Comparative Study of Multiple Testing Techniques
• Rectifying Adversarial Examples Using Their Vulnerabilities