Beneficial Perturbations Network for Defending Adversarial Examples

Abstract

Deep neural networks can be fooled by adversarial attacks: adding carefully computed small adversarial perturbations to clean inputs can cause misclassification on state-of-the-art machine learning models. The reason is that neural networks fail to accommodate the distribution drift of the input data caused by adversarial perturbations. Here, we present a new solution - Beneficial Perturbation Network (BPN) - to defend against adversarial attacks by fixing the distribution drift. During training, BPN generates and leverages beneficial perturbations (somewhat opposite to well-known adversarial perturbations) by adding new, out-of-network biasing units. Biasing units influence the parameter space of the network, to preempt and neutralize future adversarial perturbations on input data samples. To achieve this, BPN creates reverse adversarial attacks during training, with very little cost, by recycling the training gradients already computed. Reverse attacks are captured by the biasing units, and the biases can in turn effectively defend against future adversarial examples. Reverse attacks are a shortcut, i.e., they affect the network's parameters without requiring instantiation of adversarial examples that could assist training. We provide comprehensive empirical evidence showing that 1) BPN is robust to adversarial examples and is much more running memory and computationally efficient compared to classical adversarial training. 2) BPN can defend against adversarial examples with negligible additional computation and parameter costs compared to training only on clean examples; 3) BPN hurts the accuracy on clean examples much less than classic adversarial training; 4) BPN can improve the generalization of the network 5) BPN trained only with Fast Gradient Sign Attack can generalize to defend PGD attacks.

The paper is under consideration at Pattern Recognition Letters

References in corpus (13)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Language Models are Few-Shot Learners
• Improved Baselines with Momentum Contrastive Learning
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Theoretically Principled Trade-off between Robustness and Accuracy
• The Space of Transferable Adversarial Examples
• Adversarial Examples Are Not Bugs, They Are Features
• Are Labels Required for Improving Adversarial Robustness?
• Simple Black-Box Adversarial Perturbations for Deep Networks
• advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
• Adversarial Training Can Hurt Generalization
• Beneficial Perturbation Network for designing general adaptive artificial intelligence systems
• Towards Adversarial Training with Moderate Performance Improvement for Neural Network Classification

Cited by in corpus (2)

• Adversarial Defense by Latent Style Transformations
• Improving the Transferability of Adversarial Attacks on Face Recognition with Beneficial Perturbation Feature Augmentation