Feature Denoising for Improving Adversarial Robustness

Abstract

Adversarial attacks to image classification systems present challenges to convolutional networks and opportunities for understanding them. This study suggests that adversarial perturbations on images lead to noise in the features constructed by these networks. Motivated by this observation, we develop new network architectures that increase adversarial robustness by performing feature denoising. Specifically, our networks contain blocks that denoise the features using non-local means or other filters; the entire networks are trained end-to-end. When combined with adversarial training, our feature denoising networks substantially improve the state-of-the-art in adversarial robustness in both white-box and black-box attack settings. On ImageNet, under 10-iteration PGD white-box attacks where prior art has 27.9% accuracy, our method achieves 55.7%; even under extreme 2000-iteration PGD white-box attacks, our method secures 42.6% accuracy. Our method was ranked first in Competition on Adversarial Attacks and Defenses (CAAD) 2018 --- it achieved 50.6% classification accuracy on a secret, ImageNet-like test dataset against 48 unknown attackers, surpassing the runner-up approach by ~10%. Code is available at https://github.com/facebookresearch/ImageNet-Adversarial-Training.

arXiv commentCVPR 2019, code is available at: https://github.com/facebookresearch/ImageNet-Adversarial-Training

References in corpus (3)

• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Robustness May Be at Odds with Accuracy
• Evaluating and Understanding the Robustness of Adversarial Logit Pairing

Cited by in corpus (33)

• Theoretically Principled Trade-off between Robustness and Accuracy
• Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
• Dynamic Graph CNN for Learning on Point Clouds
• Natural Adversarial Examples
• Deep Closest Point: Learning Representations for Point Cloud Registration
• Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training
• Testing Robustness Against Unforeseen Adversaries
• You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
• Malware Makeover: Breaking ML-based Static Analysis by Modifying Executable Bytes
• An Alternative Surrogate Loss for PGD-based Adversarial Testing
• Black-Box Adversarial Attack with Transferable Model-based Embedding
• Defending Against Universal Attacks Through Selective Feature Regeneration
• Adversarially Robust Neural Architectures
• An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods
• Transfer of Adversarial Robustness Between Perturbation Types
• Robustness properties of Facebook's ResNeXt WSL models
• Seeing isn't Believing: Practical Adversarial Attack Against Object Detectors
• Multitask Learning Strengthens Adversarial Robustness
• Fine-grained Synthesis of Unrestricted Adversarial Examples
• Inductive Bias of Gradient Descent based Adversarial Training on Separable Data
• Attacking and Defending Machine Learning Applications of Public Cloud
• Enhancing Adversarial Defense by k-Winners-Take-All
• CorrAttack: Black-box Adversarial Attack with Structured Search
• $n$-ML: Mitigating Adversarial Examples via Ensembles of Topologically Manipulated Classifiers
• Live Face De-Identification in Video
• An empirical study of pretrained representations for few-shot classification
• Trust but Verify: An Information-Theoretic Explanation for the Adversarial Fragility of Machine Learning Systems, and a General Defense against Adversarial Attacks
• Adversarial Examples for Cost-Sensitive Classifiers
• Purifying Adversarial Perturbation with Adversarially Trained Auto-encoders
• Defending Adversarial Attacks by Correcting logits
• An Adaptive View of Adversarial Robustness from Test-time Smoothing Defense
• Bayesian Convolutional Neural Networks for Compressed Sensing Restoration
• Do Deep Minds Think Alike? Selective Adversarial Attacks for Fine-Grained Manipulation of Multiple Deep Neural Networks