Towards Defending Multiple ${\ell }_p$-norm Bounded Adversarial Perturbations via Gated Batch Normalization

Abstract

There has been extensive evidence demonstrating that deep neural networks are vulnerable to adversarial examples, which motivates the development of defenses against adversarial attacks. Existing adversarial defenses typically improve model robustness against individual specific perturbation types (\eg, ${\ell }_{\infty }$-norm bounded adversarial examples). However, adversaries are likely to generate multiple types of perturbations in practice (\eg, ${\ell }_1$, ${\ell }_2$, and ${\ell }_{\infty }$ perturbations). Some recent methods improve model robustness against adversarial attacks in multiple ${\ell }_p$ balls, but their performance against each perturbation type is still far from satisfactory. In this paper, we observe that different ${\ell }_p$ bounded adversarial perturbations induce different statistical properties that can be separated and characterized by the statistics of Batch Normalization (BN). We thus propose Gated Batch Normalization (GBN) to adversarially train a perturbation-invariant predictor for defending multiple ${\ell }_p$ bounded adversarial perturbations. GBN consists of a multi-branch BN layer and a gated sub-network. Each BN branch in GBN is in charge of one perturbation type to ensure that the normalized output is aligned towards learning perturbation-invariant representation. Meanwhile, the gated sub-network is designed to separate inputs added with different perturbation types. We perform an extensive evaluation of our approach on commonly-used dataset including MNIST, CIFAR-10, and Tiny-ImageNet, and demonstrate that GBN outperforms previous defense proposals against multiple perturbation types (\ie, ${\ell }_1$, ${\ell }_2$, and ${\ell }_{\infty }$ perturbations) by large margins.

arXiv commentAccepted on IJCV

References in corpus (15)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Explaining and Harnessing Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Revisiting Batch Normalization For Practical Domain Adaptation
• Towards the first adversarially robust neural network model on MNIST
• On Detecting Adversarial Perturbations
• A critical analysis of self-supervision, or what we can learn from a single image
• Evaluating and Understanding the Robustness of Adversarial Logit Pairing
• Testing Robustness Against Unforeseen Adversaries
• Accurate, reliable and fast robustness evaluation
• Adversarial Robustness Against the Union of Multiple Perturbation Models
• Training Robust Deep Neural Networks via Adversarial Noise Propagation
• Mode Normalization

Cited by in corpus (2)

• Mind the box: $l_1$-APGD for sparse adversarial attacks on image classifiers
• Domain Invariant Adversarial Learning