Improving White-box Robustness of Pre-processing Defenses via Joint Adversarial Training

Abstract

Deep neural networks (DNNs) are vulnerable to adversarial noise. A range of adversarial defense techniques have been proposed to mitigate the interference of adversarial noise, among which the input pre-processing methods are scalable and show great potential to safeguard DNNs. However, pre-processing methods may suffer from the robustness degradation effect, in which the defense reduces rather than improving the adversarial robustness of a target model in a white-box setting. A potential cause of this negative effect is that adversarial training examples are static and independent to the pre-processing model. To solve this problem, we investigate the influence of full adversarial examples which are crafted against the full model, and find they indeed have a positive impact on the robustness of defenses. Furthermore, we find that simply changing the adversarial training examples in pre-processing methods does not completely alleviate the robustness degradation effect. This is due to the adversarial risk of the pre-processed model being neglected, which is another cause of the robustness degradation effect. Motivated by above analyses, we propose a method called Joint Adversarial Training based Pre-processing (JATP) defense. Specifically, we formulate a feature similarity based adversarial risk for the pre-processing model by using full adversarial examples found in a feature space. Unlike standard adversarial training, we only update the pre-processing model, which prompts us to introduce a pixel-wise loss to improve its cross-model transferability. We then conduct a joint adversarial training on the pre-processing model to minimize this overall risk. Empirical results show that our method could effectively mitigate the robustness degradation effect across different target models in comparison to previous state-of-the-art approaches.

References in corpus (18)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Sequence to Sequence Learning with Neural Networks
• Theoretically Principled Trade-off between Robustness and Accuracy
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• On the Convergence and Robustness of Adversarial Training
• Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets
• MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples
• On Adaptive Attacks to Adversarial Example Defenses
• Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training
• On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses
• Residual Convolutional CTC Networks for Automatic Speech Recognition
• Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
• Black-Box Adversarial Attack with Transferable Model-based Embedding
• Adversarial Attacks on Variational Autoencoders
• Stronger and Faster Wasserstein Adversarial Attacks
• Invert and Defend: Model-based Approximate Inversion of Generative Adversarial Networks for Secure Inference