Bandlimiting Neural Networks Against Adversarial Attacks

Abstract

In this paper, we study the adversarial attack and defence problem in deep learning from the perspective of Fourier analysis. We first explicitly compute the Fourier transform of deep ReLU neural networks and show that there exist decaying but non-zero high frequency components in the Fourier spectrum of neural networks. We demonstrate that the vulnerability of neural networks towards adversarial samples can be attributed to these insignificant but non-zero high frequency components. Based on this analysis, we propose to use a simple post-averaging technique to smooth out these high frequency components to improve the robustness of neural networks against adversarial attacks. Experimental results on the ImageNet dataset have shown that our proposed method is universally effective to defend many existing adversarial attacking methods proposed in the literature, including FGSM, PGD, DeepFool and C&W attacks. Our post-averaging method is simple since it does not require any re-training, and meanwhile it can successfully defend over 95% of the adversarial samples generated by these methods without introducing any significant performance degradation (less than 1%) on the original clean images.

arXiv commentSummitted to NeurIPS 2019

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• On Evaluating Adversarial Robustness
• Detecting Adversarial Samples from Artifacts
• Defensive Distillation is Not Robust to Adversarial Examples
• MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples
• Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models
• On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses
• GenAttack: Practical Black-box Attacks with Gradient-Free Optimization
• A New Perspective on Machine Learning: How to do Perfect Supervised Learning

Cited by in corpus (1)

• Toward Few-step Adversarial Training from a Frequency Perspective