On the Structural Sensitivity of Deep Convolutional Networks to the Directions of Fourier Basis Functions
arXiv:1809.04098
Abstract
Data-agnostic quasi-imperceptible perturbations on inputs are known to degrade recognition accuracy of deep convolutional networks severely. This phenomenon is considered to be a potential security issue. Moreover, some results on statistical generalization guarantees indicate that the phenomenon can be a key to improve the networks' generalization. However, the characteristics of the shared directions of such harmful perturbations remain unknown. Our primal finding is that convolutional networks are sensitive to the directions of Fourier basis functions. We derived the property by specializing a hypothesis of the cause of the sensitivity, known as the linearity of neural networks, to convolutional networks and empirically validated it. As a by-product of the analysis, we propose an algorithm to create shift-invariant universal adversarial perturbations available in black-box settings.
arXiv commentCVPR 2019
References in corpus (17)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Ensemble Adversarial Training: Attacks and Defenses
- Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
- Provable defenses against adversarial examples via the convex outer adversarial polytope
- The Space of Transferable Adversarial Examples
- Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
- A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
- A study of the effect of JPG compression on adversarial images
- Measuring the tendency of CNNs to Learn Surface Statistical Regularities
- Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
- The Singular Values of Convolutional Layers
- Fast Feature Fool: A data independent approach to universal adversarial perturbations
- Robustness of classifiers to universal perturbations: a geometric perspective
- Defense Against Adversarial Attacks with Saak Transform
- Art of singular vectors and universal adversarial perturbations