Interpreting Adversarially Trained Convolutional Neural Networks

Abstract

We attempt to interpret how adversarially trained convolutional neural networks (AT-CNNs) recognize objects. We design systematic approaches to interpret AT-CNNs in both qualitative and quantitative ways and compare them with normally trained models. Surprisingly, we find that adversarial training alleviates the texture bias of standard CNNs when trained on object recognition tasks, and helps CNNs learn a more shape-biased representation. We validate our hypothesis from two aspects. First, we compare the salience maps of AT-CNNs and standard CNNs on clean images and images under different transformations. The comparison could visually show that the prediction of the two types of CNNs is sensitive to dramatically different types of features. Second, to achieve quantitative verification, we construct additional test datasets that destroy either textures or shapes, such as style-transferred version of clean data, saturated images and patch-shuffled ones, and then evaluate the classification accuracy of AT-CNNs and normal CNNs on these datasets. Our findings shed some light on why AT-CNNs are more robust than those normally trained ones and contribute to a better understanding of adversarial training over CNNs from an interpretation perspective.

To apper in ICML19

Cited by in corpus (32)

• Do Adversarially Robust ImageNet Models Transfer Better?
• Adversarial Robustness as a Prior for Learned Representations
• Boosting Adversarial Training with Hypersphere Embedding
• Partial success in closing the gap between human and machine vision
• Robust Local Features for Improving the Generalization of Adversarial Training
• Adversarial Examples in Modern Machine Learning: A Review
• Adversarial Examples Improve Image Recognition
• Attention Meets Perturbations: Robust and Interpretable Attention with Adversarial Training
• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• Boundary thickness and robustness in learning models
• Robust and Generalizable Visual Representation Learning via Random Convolutions
• Achieving Adversarial Robustness via Sparsity
• Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learning
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training
• Robust and Accurate Object Detection via Adversarial Learning
• Recent Advances in Understanding Adversarial Robustness of Deep Neural Networks
• Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better
• Improving the robustness of ImageNet classifiers using elements of human visual cognition
• Amata: An Annealing Mechanism for Adversarial Training Acceleration
• Understanding and Diagnosing Vulnerability under Adversarial Attacks
• Natural Perturbed Training for General Robustness of Neural Network Classifiers
• Relevance Attack on Detectors
• An Experimental Study of Semantic Continuity for Deep Learning Models
• Semantics-Preserving Adversarial Training
• Built-in Elastic Transformations for Improved Robustness
• Attack to Fool and Explain Deep Networks
• Noise Modulation: Let Your Model Interpret Itself
• Shape Defense Against Adversarial Attacks
• Spatiotemporal Attacks for Embodied Agents
• Understanding the robustness of deep neural network classifiers for breast cancer screening
• HRFA: High-Resolution Feature-based Attack