Robust Models Are More Interpretable Because Attributions Look Normal

Abstract

Recent work has found that adversarially-robust deep networks used for image classification are more interpretable: their feature attributions tend to be sharper, and are more concentrated on the objects associated with the image's ground-truth class. We show that smooth decision boundaries play an important role in this enhanced interpretability, as the model's input gradients around data points will more closely align with boundaries' normal vectors when they are smooth. Thus, because robust models have smoother boundaries, the results of gradient-based attribution methods, like Integrated Gradients and DeepLift, will capture more accurate information about nearby decision boundaries. This understanding of robust interpretability leads to our second contribution: \emph{boundary attributions}, which aggregate information about the normal vectors of local decision boundaries to explain a classification outcome. We show that by leveraging the key factors underpinning robust interpretability, boundary attributions produce sharper, more concentrated visual explanations -- even on non-robust models. Any example implementation can be found at \url{https://github.com/zifanw/boundary}.

References in corpus (22)

• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Axiomatic Attribution for Deep Networks
• Striving for Simplicity: The All Convolutional Net
• Learning Important Features Through Propagating Activation Differences
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• SmoothGrad: removing noise by adding noise
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Captum: A unified and generic model interpretability library for PyTorch
• Certified Adversarial Robustness via Randomized Smoothing
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Sanity Checks for Saliency Maps
• Evaluating Robustness of Neural Networks with Mixed Integer Programming
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Explaining Image Classifiers by Counterfactual Generation
• How Important Is a Neuron?
• Provable Robustness of ReLU networks via Maximization of Linear Regions
• Robust Attribution Regularization
• On the Connection Between Adversarial Robustness and Saliency Map Interpretability
• Randomized Smoothing of All Shapes and Sizes
• Smoothed Geometry for Robust Attribution
• Provable Certificates for Adversarial Examples: Fitting a Ball in the Union of Polytopes
• On Locality of Local Explanation Models