Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients

Abstract

Deep neural networks have proven remarkably effective at solving many classification problems, but have been criticized recently for two major weaknesses: the reasons behind their predictions are uninterpretable, and the predictions themselves can often be fooled by small adversarial perturbations. These problems pose major obstacles for the adoption of neural networks in domains that require security or transparency. In this work, we evaluate the effectiveness of defenses that differentiably penalize the degree to which small changes in inputs can alter model predictions. Across multiple attacks, architectures, defenses, and datasets, we find that neural networks trained with this input gradient regularization exhibit robustness to transferred adversarial examples generated to fool all of the other models. We also find that adversarial examples generated to fool gradient-regularized models fool all other models equally well, and actually lead to more "legitimate," interpretable misclassifications as rated by people (which we confirm in a human subject experiment). Finally, we demonstrate that regularizing input gradients makes them more naturally interpretable as rationales for model predictions. We conclude by discussing this relationship between interpretability and robustness in deep neural networks.

arXiv commentTo appear in AAAI 2018

Cited by in corpus (122)

• Knowledge Distillation: A Survey
• Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems
• Robustness May Be at Odds with Accuracy
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• Adversarial Examples: Opportunities and Challenges
• Adversarial Weight Perturbation Helps Robust Generalization
• On the Convergence and Robustness of Adversarial Training
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks
• Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks
• MMA Training: Direct Input Space Margin Maximization through Adversarial Training
• Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
• A Closer Look at Accuracy vs. Robustness
• Hardening Random Forest Cyber Detectors Against Adversarial Attacks
• Interpretation of Neural Networks is Fragile
• On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
• On the (In)fidelity and Sensitivity for Explanations
• Security and Privacy Issues in Deep Learning
• Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attack
• Diversity can be Transferred: Output Diversification for White- and Black-box Attacks
• Understanding and Improving Fast Adversarial Training
• Recent Advances in Adversarial Training for Adversarial Robustness
• Stakeholders in Explainable AI
• Robust Learning with Jacobian Regularization
• Interpretations are useful: penalizing explanations to align neural networks with prior knowledge
• Do Wider Neural Networks Really Help Adversarial Robustness?
• Deep Gamblers: Learning to Abstain with Portfolio Theory
• L2-Nonexpansive Neural Networks
• White Box Methods for Explanations of Convolutional Neural Networks in Image Classification Tasks
• Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey
• Bridging Adversarial Robustness and Gradient Interpretability
• What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play
• Investigating the significance of adversarial attacks and their relation to interpretability for radar-based human activity recognition systems
• Adversarial Detection of Flash Malware: Limitations and Open Issues
• Defense Against Adversarial Attacks with Saak Transform
• Quantifying the Preferential Direction of the Model Gradient in Adversarial Training With Projected Gradient Descent
• Brain Programming is Immune to Adversarial Attacks: Towards Accurate and Robust Image Classification using Symbolic Learning
• Improving adversarial robustness of deep neural networks by using semantic information
• Synthetic Benchmarks for Scientific Research in Explainable Machine Learning
• FenceBox: A Platform for Defeating Adversarial Examples with Data Augmentation Techniques
• ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies
• Mitigating Advanced Adversarial Attacks with More Advanced Gradient Obfuscation Techniques
• Attacking Adversarial Attacks as A Defense
• EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attacks
• Gradient Band-based Adversarial Training for Generalized Attack Immunity of A3C Path Finding
• Interpreting and Evaluating Neural Network Robustness
• Towards Security Threats of Deep Learning Systems: A Survey
• Deep Neural Networks for Choice Analysis: A Statistical Learning Theory Perspective
• AIR5: Five Pillars of Artificial Intelligence Research
• Jacobian Adversarially Regularized Networks for Robustness
• Towards Robust Training of Neural Networks by Regularizing Adversarial Gradients
• Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps
• The Geometry of Adversarial Training in Binary Classification
• Rethinking the Role of Gradient-Based Attribution Methods for Model Interpretability
• Model-Based Domain Generalization
• Second Order Optimization for Adversarial Robustness and Interpretability
• Why Adversarial Reprogramming Works, When It Fails, and How to Tell the Difference
• Enhancing Adversarial Defense by k-Winners-Take-All
• Efficient Two-Step Adversarial Defense for Deep Neural Networks
• Improving Global Adversarial Robustness Generalization With Adversarially Trained GAN
• ATMPA: Attacking Machine Learning-based Malware Visualization Detection Methods via Adversarial Examples
• Blind Adversarial Network Perturbations
• Defense-VAE: A Fast and Accurate Defense against Adversarial Attacks
• Adversarial Learning with Margin-based Triplet Embedding Regularization
• Deep Neural Networks for Choice Analysis: Extracting Complete Economic Information for Interpretation
• Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers
• From Shallow to Deep Interactions Between Knowledge Representation, Reasoning and Machine Learning (Kay R. Amel group)
• Dealing with Adversarial Player Strategies in the Neural Network Game iNNk through Ensemble Learning
• Double Backpropagation for Training Autoencoders against Adversarial Attack
• Knowledge Distillation as Semiparametric Inference
• Gradient-based Analysis of NLP Models is Manipulable
• Input Hessian Regularization of Neural Networks
• Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attacks
• Connecting Interpretability and Robustness in Decision Trees through Separation
• CAP: Co-Adversarial Perturbation on Weights and Features for Improving Generalization of Graph Neural Networks
• Exposing the Robustness and Vulnerability of Hybrid 8T-6T SRAM Memory Architectures to Adversarial Attacks in Deep Neural Networks
• MPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs
• Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizations
• Medical Aegis: Robust adversarial protectors for medical images
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• Exploring Transferable and Robust Adversarial Perturbation Generation from the Perspective of Network Hierarchy
• FineFool: Fine Object Contour Attack via Attention
• Adversarial Robustness for Unsupervised Domain Adaptation
• On Procedural Adversarial Noise Attack And Defense
• Progressive Defense Against Adversarial Attacks for Deep Learning as a Service in Internet of Things
• Adversarial Feature Desensitization
• On the human-recognizability phenomenon of adversarially trained deep image classifiers
• Understanding the Error in Evaluating Adversarial Robustness
• Disentangled Deep Autoencoding Regularization for Robust Image Classification
• An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness
• What it Thinks is Important is Important: Robustness Transfers through Input Gradients
• Towards Robust Classification Model by Counterfactual and Invariant Data Generation
• The Definitions of Interpretability and Learning of Interpretable Models
• Explaining Adversarial Vulnerability with a Data Sparsity Hypothesis
• Do Input Gradients Highlight Discriminative Features?
• Enhance CNN Robustness Against Noises for Classification of 12-Lead ECG with Variable Length
• Deep Neural Networks for Choice Analysis: Architectural Design with Alternative-Specific Utility Functions
• Rethinking Empirical Evaluation of Adversarial Robustness Using First-Order Attack Methods
• The Outer Product Structure of Neural Network Derivatives
• Why Blocking Targeted Adversarial Perturbations Impairs the Ability to Learn
• Robustness from Simple Classifiers
• Dual Head Adversarial Training
• MadNet: Using a MAD Optimization for Defending Against Adversarial Attacks
• Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses
• Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search
• Noise Modulation: Let Your Model Interpret Itself
• Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning
• Adversarial Multi-scale Feature Learning for Person Re-identification
• Defense Through Diverse Directions
• Semantics-Preserving Adversarial Training
• Constraining Logits by Bounded Function for Adversarial Robustness
• Deep Minimax Probability Machine
• Recent Advances in Large Margin Learning
• Deep Adaptive Wavelet Network
• Bridged Adversarial Training
• Training Efficiency and Robustness in Deep Learning
• Identifying and Exploiting Structures for Reliable Deep Learning
• Information-Corrected Estimation: A Generalization Error Reducing Parameter Estimation Method
• Improving Compositionality of Neural Networks by Decoding Representations to Inputs
• On Connections between Regularizations for Improving DNN Robustness
• Orthogonal Deep Models As Defense Against Black-Box Attacks
• Improving Resistance to Adversarial Deformations by Regularizing Gradients