Interpretability is a Kind of Safety: An Interpreter-based Ensemble for Adversary Defense

Abstract

While having achieved great success in rich real-life applications, deep neural network (DNN) models have long been criticized for their vulnerability to adversarial attacks. Tremendous research efforts have been dedicated to mitigating the threats of adversarial attacks, but the essential trait of adversarial examples is not yet clear, and most existing methods are yet vulnerable to hybrid attacks and suffer from counterattacks. In light of this, in this paper, we first reveal a gradient-based correlation between sensitivity analysis-based DNN interpreters and the generation process of adversarial examples, which indicates the Achilles's heel of adversarial attacks and sheds light on linking together the two long-standing challenges of DNN: fragility and unexplainability. We then propose an interpreter-based ensemble framework called X-Ensemble for robust adversary defense. X-Ensemble adopts a novel detection-rectification process and features in building multiple sub-detectors and a rectifier upon various types of interpretation information toward target classifiers. Moreover, X-Ensemble employs the Random Forests (RF) model to combine sub-detectors into an ensemble detector for adversarial hybrid attacks defense. The non-differentiable property of RF further makes it a precious choice against the counterattack of adversaries. Extensive experiments under various types of state-of-the-art attacks and diverse attack scenarios demonstrate the advantages of X-Ensemble to competitive baseline methods.

arXiv comment10 pages, accepted to KDD'20

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Explaining and Harnessing Adversarial Examples
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Intriguing properties of neural networks
• Axiomatic Attribution for Deep Networks
• Striving for Simplicity: The All Convolutional Net
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
• Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
• MagNet: a Two-Pronged Defense against Adversarial Examples
• AlphaStock: A Buying-Winners-and-Selling-Losers Investment Strategy using Interpretable Deep Reinforcement Attention Networks
• A New Defense Against Adversarial Images: Turning a Weakness into a Strength
• Saliency Methods for Explaining Adversarial Attacks

Cited by in corpus (5)

• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• SoK: Modeling Explainability in Security Analytics for Interpretability, Trustworthiness, and Usability
• Adversarial Attacks and Defenses: An Interpretation Perspective
• PASA: Attack Agnostic Unsupervised Adversarial Detection using Prediction & Attribution Sensitivity Analysis
• ExAD: An Ensemble Approach for Explanation-based Adversarial Detection