When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures

Abstract

State-of-the-art deep neural networks (DNNs) are highly effective in solving many complex real-world problems. However, these models are vulnerable to adversarial perturbation attacks, and despite the plethora of research in this domain, to this day, adversaries still have the upper hand in the cat and mouse game of adversarial example generation methods vs. detection and prevention methods. In this research, we present a novel detection method that uses Shapley Additive Explanations (SHAP) values computed for the internal layers of a DNN classifier to discriminate between normal and adversarial inputs. We evaluate our method by building an extensive dataset of adversarial examples over the popular CIFAR-10 and MNIST datasets, and training a neural network-based detector to distinguish between normal and adversarial inputs. We evaluate our detector against adversarial examples generated by diverse state-of-the-art attacks and demonstrate its high detection accuracy and strong generalization ability to adversarial inputs generated with different attack methods.

References in corpus (14)

• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• Learning Important Features Through Propagating Activation Differences
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Synthesizing Robust Adversarial Examples
• On Evaluating Adversarial Robustness
• Adversarial Examples Are Not Bugs, They Are Features
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Adversarial Machine Learning at Scale
• Detecting Adversarial Samples from Artifacts
• On the (Statistical) Detection of Adversarial Examples
• A study of the effect of JPG compression on adversarial images
• Adaptive Gradient Methods with Dynamic Bound of Learning Rate
• A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance

Cited by in corpus (6)

• Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research
• What Do You See? Evaluation of Explainable Artificial Intelligence (XAI) Interpretability through Neural Backdoors
• RAID: Randomized Adversarial-Input Detection for Neural Networks
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
• Towards an Explanation Space to Align Humans and Explainable-AI Teamwork