Training Robust Deep Neural Networks via Adversarial Noise Propagation

Abstract

In practice, deep neural networks have been found to be vulnerable to various types of noise, such as adversarial examples and corruption. Various adversarial defense methods have accordingly been developed to improve adversarial robustness for deep models. However, simply training on data mixed with adversarial examples, most of these models still fail to defend against the generalized types of noise. Motivated by the fact that hidden layers play a highly important role in maintaining a robust model, this paper proposes a simple yet powerful training algorithm, named \emph{Adversarial Noise Propagation} (ANP), which injects noise into the hidden layers in a layer-wise manner. ANP can be implemented efficiently by exploiting the nature of the backward-forward training style. Through thorough investigations, we determine that different hidden layers make different contributions to model robustness and clean accuracy, while shallow layers are comparatively more critical than deep layers. Moreover, our framework can be easily combined with other adversarial training methods to further improve model robustness by exploiting the potential of hidden layers. Extensive experiments on MNIST, CIFAR-10, CIFAR-10-C, CIFAR-10-P, and ImageNet demonstrate that ANP enables the strong robustness for deep models against both adversarial and corrupted ones, and also significantly outperforms various adversarial defense methods.

arXiv comment14 pages

References in corpus (14)

• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Ensemble Adversarial Training: Attacks and Defenses
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Theoretically Principled Trade-off between Robustness and Accuracy
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• On Evaluating Adversarial Robustness
• Adversarially Robust Generalization Requires More Data
• Robustness May Be at Odds with Accuracy
• Interpreting and Improving Adversarial Robustness of Deep Neural Networks with Neuron Sensitivity
• Are All Layers Created Equal?
• Image Synthesis with a Single (Robust) Classifier
• Deep Defense: Training DNNs with Improved Adversarial Robustness
• Accurate, reliable and fast robustness evaluation
• Learning Adversary-Resistant Deep Neural Networks

Cited by in corpus (9)

• Binary Neural Networks: A Survey
• Interpreting and Improving Adversarial Robustness of Deep Neural Networks with Neuron Sensitivity
• A Comprehensive Evaluation Framework for Deep Model Robustness
• Towards Defending Multiple ${\ell }_p$-norm Bounded Adversarial Perturbations via Gated Batch Normalization
• Improving Model Robustness with Latent Distribution Locally and Globally
• Smoothed Inference for Adversarially-Trained Models
• Colored Noise Injection for Training Adversarially Robust Neural Networks
• Challenging the adversarial robustness of DNNs based on error-correcting output codes
• What Can Style Transfer and Paintings Do For Model Robustness?