A Boundary Tilting Persepective on the Phenomenon of Adversarial Examples
arXiv:1608.07690
Abstract
Deep neural networks have been shown to suffer from a surprising weakness: their classification outputs can be changed by small, non-random perturbations of their inputs. This adversarial example phenomenon has been explained as originating from deep networks being "too linear" (Goodfellow et al., 2014). We show here that the linear explanation of adversarial examples presents a number of limitations: the formal argument is not convincing, linear classifiers do not always suffer from the phenomenon, and when they do their adversarial examples are different from the ones affecting deep networks. We propose a new perspective on the phenomenon. We argue that adversarial examples exist when the classification boundary lies close to the submanifold of sampled data, and present a mathematical analysis of this new perspective in the linear case. We define the notion of adversarial strength and show that it can be reduced to the deviation angle between the classifier considered and the nearest centroid classifier. Then, we show that the adversarial strength can be made arbitrarily high independently of the classification performance due to a mechanism that we call boundary tilting. This result leads us to defining a new taxonomy of adversarial examples. Finally, we show that the adversarial strength observed in practice is directly dependent on the level of regularisation used and the strongest adversarial examples, symptomatic of overfitting, can be avoided by using a proper level of regularisation.
arXiv commentarXiv admin note: text overlap with arXiv:1412.6572 by other authors
References in corpus (2)
Cited by in corpus (65)
- Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
- Adversarial Examples Are Not Bugs, They Are Features
- Detecting Adversarial Samples from Artifacts
- Adversarial Examples: Attacks and Defenses for Deep Learning
- Understanding Measures of Uncertainty for Adversarial Example Detection
- Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
- Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training
- A Review of Adversarial Attack and Defense for Classification Methods
- Label-Only Membership Inference Attacks
- Batch Normalization is a Cause of Adversarial Vulnerability
- Adversarial Examples - A Complete Characterisation of the Phenomenon
- Sufficient Conditions for Idealised Models to Have No Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks
- Bridging Adversarial Robustness and Gradient Interpretability
- Defense against Universal Adversarial Perturbations
- Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics
- Provably scale-covariant continuous hierarchical networks based on scale-normalized differential expressions coupled in cascade
- High Dimensional Spaces, Deep Learning and Adversarial Examples
- Boundary thickness and robustness in learning models
- Quantifying the Preferential Direction of the Model Gradient in Adversarial Training With Projected Gradient Descent
- Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales
- A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning
- Adversarial Training Versus Weight Decay
- Towards Understanding Adversarial Examples Systematically: Exploring Data Size, Task and Model Factors
- ML-LOO: Detecting Adversarial Examples with Feature Attribution
- Adversarial Attacks and Defense on Texts: A Survey
- Adversarial Examples on Object Recognition: A Comprehensive Survey
- Adversarial Robustness via Label-Smoothing
- Data from Model: Extracting Data from Non-robust and Robust Models
- Sparse Adversarial Video Attacks with Spatial Transformations
- Robustness of classifiers to uniform and Gaussian noise
- Investigating Decision Boundaries of Trained Neural Networks
- Softmax-based Classification is k-means Clustering: Formal Proof, Consequences for Adversarial Attacks, and Improvement through Centroid Based Tailoring
- On the Need for Topology-Aware Generative Models for Manifold-Based Defenses
- The Intriguing Relation Between Counterfactual Explanations and Adversarial Examples
- Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
- On the Effect of Low-Rank Weights on Adversarial Robustness of Neural Networks
- A Bayes-Optimal View on Adversarial Examples
- The Adversarial Attack and Detection under the Fisher Information Metric
- Random Directional Attack for Fooling Deep Neural Networks
- Stochastic Combinatorial Ensembles for Defending Against Adversarial Examples
- Understanding Adversarial Examples from the Mutual Influence of Images and Perturbations
- Universal Adversarial Perturbations Through the Lens of Deep Steganography: Towards A Fourier Perspective
- Predicting Adversarial Examples with High Confidence
- Trust but Verify: An Information-Theoretic Explanation for the Adversarial Fragility of Machine Learning Systems, and a General Defense against Adversarial Attacks
- PhysGAN: Generating Physical-World-Resilient Adversarial Examples for Autonomous Driving
- Computing Linear Restrictions of Neural Networks
- Adversarial Risk Bounds for Neural Networks through Sparsity based Compression
- FineFool: Fine Object Contour Attack via Attention
- Visually Imperceptible Adversarial Patch Attacks on Digital Images
- Adversarial Robustness Guarantees for Random Deep Neural Networks
- ROBY: Evaluating the Robustness of a Deep Model by its Decision Boundaries
- What Do Adversarially Robust Models Look At?
- An Information-Theoretic Explanation for the Adversarial Fragility of AI Classifiers
- Do Input Gradients Highlight Discriminative Features?
- Who is Responsible for Adversarial Defense?
- Heat and Blur: An Effective and Fast Defense Against Adversarial Examples
- CD-UAP: Class Discriminative Universal Adversarial Perturbation
- Verifying the Causes of Adversarial Examples
- Adversarial Data Encryption
- Towards Explaining Adversarial Examples Phenomenon in Artificial Neural Networks
- Analyzing Adversarial Robustness of Deep Neural Networks in Pixel Space: a Semantic Perspective
- Hard-label Manifolds: Unexpected Advantages of Query Efficiency for Finding On-manifold Adversarial Examples
- Towards A Conceptually Simple Defensive Approach for Few-shot classifiers Against Adversarial Support Samples
- A novel network training approach for open set image recognition
- Learning Task-aware Robust Deep Learning Systems