Identifying Layers Susceptible to Adversarial Attacks

Abstract

In this paper, we investigate the use of pretraining with adversarial networks, with the objective of discovering the relationship between network depth and robustness. For this purpose, we selectively retrain different portions of VGG and ResNet architectures on CIFAR-10, Imagenette, and ImageNet using non-adversarial and adversarial data. Experimental results show that susceptibility to adversarial samples is associated with low-level feature extraction layers. Therefore, retraining of high-level layers is insufficient for achieving robustness. Furthermore, adversarial attacks yield outputs from early layers that differ statistically from features for non-adversarial samples and do not permit consistent classification by subsequent layers. This supports common hypotheses regarding the association of robustness with the feature extractor, insufficiency of deeper layers in providing robustness, and large differences in adversarial and non-adversarial feature vectors.

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Understanding Neural Networks Through Deep Visualization
• Theoretically Principled Trade-off between Robustness and Accuracy
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Adversarial Examples Are Not Bugs, They Are Features
• Are All Layers Created Equal?
• Smooth Adversarial Training
• Metric Learning for Adversarial Robustness
• Improving Adversarial Robustness via Channel-wise Activation Suppressing
• High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
• A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning