Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients

Abstract

Deep neural networks have proven remarkably effective at solving many classification problems, but have been criticized recently for two major weaknesses: the reasons behind their predictions are uninterpretable, and the predictions themselves can often be fooled by small adversarial perturbations. These problems pose major obstacles for the adoption of neural networks in domains that require security or transparency. In this work, we evaluate the effectiveness of defenses that differentiably penalize the degree to which small changes in inputs can alter model predictions. Across multiple attacks, architectures, defenses, and datasets, we find that neural networks trained with this input gradient regularization exhibit robustness to transferred adversarial examples generated to fool all of the other models. We also find that adversarial examples generated to fool gradient-regularized models fool all other models equally well, and actually lead to more "legitimate," interpretable misclassifications as rated by people (which we confirm in a human subject experiment). Finally, we demonstrate that regularizing input gradients makes them more naturally interpretable as rationales for model predictions. We conclude by discussing this relationship between interpretability and robustness in deep neural networks.

To appear in AAAI 2018

Cited by in corpus (55)

• On the Convergence and Robustness of Adversarial Training
• Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
• On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
• Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attack
• Recent Advances in Adversarial Training for Adversarial Robustness
• Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey
• Bridging Adversarial Robustness and Gradient Interpretability
• Investigating the significance of adversarial attacks and their relation to interpretability for radar-based human activity recognition systems
• Synthetic Benchmarks for Scientific Research in Explainable Machine Learning
• FenceBox: A Platform for Defeating Adversarial Examples with Data Augmentation Techniques
• Mitigating Advanced Adversarial Attacks with More Advanced Gradient Obfuscation Techniques
• EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attacks
• Attacking Adversarial Attacks as A Defense
• Interpreting and Evaluating Neural Network Robustness
• Jacobian Adversarially Regularized Networks for Robustness
• AIR5: Five Pillars of Artificial Intelligence Research
• Improving Global Adversarial Robustness Generalization With Adversarially Trained GAN
• Second Order Optimization for Adversarial Robustness and Interpretability
• From Shallow to Deep Interactions Between Knowledge Representation, Reasoning and Machine Learning (Kay R. Amel group)
• Blind Adversarial Network Perturbations
• Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers
• Dealing with Adversarial Player Strategies in the Neural Network Game iNNk through Ensemble Learning
• Knowledge Distillation as Semiparametric Inference
• Input Hessian Regularization of Neural Networks
• Double Backpropagation for Training Autoencoders against Adversarial Attack
• CAP: Co-Adversarial Perturbation on Weights and Features for Improving Generalization of Graph Neural Networks
• Exposing the Robustness and Vulnerability of Hybrid 8T-6T SRAM Memory Architectures to Adversarial Attacks in Deep Neural Networks
• Connecting Interpretability and Robustness in Decision Trees through Separation
• Disentangled Deep Autoencoding Regularization for Robust Image Classification
• On Procedural Adversarial Noise Attack And Defense
• On the human-recognizability phenomenon of adversarially trained deep image classifiers
• Exploring Transferable and Robust Adversarial Perturbation Generation from the Perspective of Network Hierarchy
• Adversarial Robustness for Unsupervised Domain Adaptation
• Understanding the Error in Evaluating Adversarial Robustness
• Robustness from Simple Classifiers
• Rethinking Empirical Evaluation of Adversarial Robustness Using First-Order Attack Methods
• Why Blocking Targeted Adversarial Perturbations Impairs the Ability to Learn
• Towards Robust Classification Model by Counterfactual and Invariant Data Generation
• The Definitions of Interpretability and Learning of Interpretable Models
• On Connections between Regularizations for Improving DNN Robustness
• Orthogonal Deep Models As Defense Against Black-Box Attacks
• Semantics-Preserving Adversarial Training
• Dual Head Adversarial Training
• Noise Modulation: Let Your Model Interpret Itself
• Constraining Logits by Bounded Function for Adversarial Robustness
• Deep Minimax Probability Machine
• Deep Adaptive Wavelet Network
• Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning
• Adversarial Multi-scale Feature Learning for Person Re-identification
• Training Efficiency and Robustness in Deep Learning
• Bridged Adversarial Training
• Identifying and Exploiting Structures for Reliable Deep Learning
• Improving Compositionality of Neural Networks by Decoding Representations to Inputs
• Improving Resistance to Adversarial Deformations by Regularizing Gradients
• Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses