Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?

Abstract

For a standard convolutional neural network, optimizing over the input pixels to maximize the score of some target class will generally produce a grainy-looking version of the original image. However, Santurkar et al. (2019) demonstrated that for adversarially-trained neural networks, this optimization produces images that uncannily resemble the target class. In this paper, we show that these "perceptually-aligned gradients" also occur under randomized smoothing, an alternative means of constructing adversarially-robust classifiers. Our finding supports the hypothesis that perceptually-aligned gradients may be a general property of robust classifiers. We hope that our results will inspire research aimed at explaining this link between perceptually-aligned gradients and adversarial robustness.

arXiv commentTo appear in the "Science Meets Engineering of Deep Learning" Workshop at NeurIPS 2019

References in corpus (5)

• Certified Adversarial Robustness via Randomized Smoothing
• A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
• Adversarial Robustness as a Prior for Learned Representations
• Defending against Whitebox Adversarial Attacks via Randomized Discretization
• Certifiably Robust Interpretation in Deep Learning

Cited by in corpus (13)

• The Origins and Prevalence of Texture Bias in Convolutional Neural Networks
• RobustBench: a standardized adversarial robustness benchmark
• Do Adversarially Robust ImageNet Models Transfer Better?
• Partial success in closing the gap between human and machine vision
• Quantifying the Preferential Direction of the Model Gradient in Adversarial Training With Projected Gradient Descent
• Poisoned classifiers are not only backdoored, they are fundamentally broken
• Rethinking the Role of Gradient-Based Attribution Methods for Model Interpretability
• On the Benefits of Models with Perceptually-Aligned Gradients
• On the human-recognizability phenomenon of adversarially trained deep image classifiers
• Smoothed Inference for Adversarially-Trained Models
• Bio-inspired Robustness: A Review
• Energy Attack: On Transferring Adversarial Examples
• Combining Diverse Feature Priors