Adversarial Attacks and Defenses: An Interpretation Perspective
arXiv:2004.11488
Abstract
Despite the recent advances in a wide spectrum of applications, machine learning models, especially deep neural networks, have been shown to be vulnerable to adversarial attacks. Attackers add carefully-crafted perturbations to input, where the perturbations are almost imperceptible to humans, but can cause models to make wrong predictions. Techniques to protect models against adversarial input are called adversarial defense methods. Although many approaches have been proposed to study adversarial attacks and defenses in different scenarios, an intriguing and crucial challenge remains that how to really understand model vulnerability? Inspired by the saying that "if you know yourself and your enemy, you need not fear the battles", we may tackle the aforementioned challenge after interpreting machine learning models to open the black-boxes. The goal of model interpretation, or interpretable machine learning, is to extract human-understandable terms for the working mechanism of models. Recently, some approaches start incorporating interpretation into the exploration of adversarial attacks and defenses. Meanwhile, we also observe that many existing methods of adversarial attacks and defenses, although not explicitly claimed, can be understood from the perspective of interpretation. In this paper, we review recent work on adversarial attacks and defenses, particularly from the perspective of machine learning interpretation. We categorize interpretation into two types, feature-level interpretation and model-level interpretation. For each type of interpretation, we elaborate on how it could be used for adversarial attacks and defenses. We then briefly illustrate additional correlations between interpretation and adversaries. Finally, we discuss the challenges and future directions along tackling adversary issues with interpretation.
References in corpus (45)
- Towards A Rigorous Science of Interpretable Machine Learning
- Axiomatic Attribution for Deep Networks
- Methods for Interpreting and Understanding Deep Neural Networks
- Ensemble Adversarial Training: Attacks and Defenses
- Interpretable machine learning: definitions, methods, and applications
- Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
- Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
- Recent Trends in Deep Learning Based Natural Language Processing
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- Adversarial Attacks on Neural Networks for Graph Data
- BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
- Theoretically Principled Trade-off between Robustness and Accuracy
- SmoothGrad: removing noise by adding noise
- This Looks Like That: Deep Learning for Interpretable Image Recognition
- Adversarially Robust Generalization Requires More Data
- Understanding Neural Networks through Representation Erasure
- Adversarial Examples Are Not Bugs, They Are Features
- Adversarial Machine Learning at Scale
- Detecting Adversarial Samples from Artifacts
- On the (Statistical) Detection of Adversarial Examples
- Robustness May Be at Odds with Accuracy
- A Survey of Learning Causality with Data: Problems and Methods
- On Detecting Adversarial Perturbations
- Mitigating Adversarial Effects Through Randomization
- Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives
- Fooling Neural Network Interpretations via Adversarial Model Manipulation
- Distilling Knowledge from Deep Networks with Applications to Healthcare Domain
- XFake: Explainable Fake News Detector with Visualizations
- Learning Disentangled Representations for Recommendation
- How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
- Image Synthesis with a Single (Robust) Classifier
- Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
- Interpreting Adversarially Trained Convolutional Neural Networks
- Unrestricted Adversarial Examples
- Restricting the Flow: Information Bottlenecks for Attribution
- Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples
- Causal Learning and Explanation of Deep Neural Networks via Autoencoded Activations
- Adversarial Examples Improve Image Recognition
- Explaining Deep Learning Models using Causal Inference
- Deep Leakage from Gradients
- When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures
- Certifiably Robust Interpretation in Deep Learning
- ML-LOO: Detecting Adversarial Examples with Feature Attribution
- Interpretability is a Kind of Safety: An Interpreter-based Ensemble for Adversary Defense
- Are Interpretations Fairly Evaluated? A Definition Driven Pipeline for Post-Hoc Interpretability