On the Geometry of Adversarial Examples

Abstract

Adversarial examples are a pervasive phenomenon of machine learning models where seemingly imperceptible perturbations to the input lead to misclassifications for otherwise statistically accurate models. We propose a geometric framework, drawing on tools from the manifold reconstruction literature, to analyze the high-dimensional geometry of adversarial examples. In particular, we highlight the importance of codimension: for low-dimensional data manifolds embedded in high-dimensional space there are many directions off the manifold in which to construct adversarial examples. Adversarial examples are a natural consequence of learning a decision boundary that classifies the low-dimensional data manifold well, but classifies points near the manifold incorrectly. Using our geometric framework we prove (1) a tradeoff between robustness under different norms, (2) that adversarial training in balls around the data is sample inefficient, and (3) sufficient sampling conditions under which nearest neighbor classifiers and ball-based adversarial training are robust.

arXiv commentImprovements to clarity and presentation over initial submission

References in corpus (12)

• Explaining and Harnessing Adversarial Examples
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Intriguing properties of neural networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• The Marginal Value of Adaptive Gradient Methods in Machine Learning
• Adversarially Robust Generalization Requires More Data
• Large-Margin Softmax Loss for Convolutional Neural Networks
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Analyzing the Robustness of Nearest Neighbors to Adversarial Examples
• Learning Contact-Rich Manipulation Skills with Guided Policy Search
• Distributionally Adversarial Attack

Cited by in corpus (18)

• Adversarial Policies: Attacking Deep Reinforcement Learning
• Adversarial Explanations for Understanding Image Classification Decisions and Improved Neural Network Robustness
• Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
• Adversarial Examples Make Strong Poisons
• Robustness of Bayesian Neural Networks to Gradient-Based Attacks
• Potential adversarial samples for white-box attacks
• A Provable Defense for Deep Residual Networks
• How adversarial attacks can disrupt seemingly stable accurate classifiers
• Distributionally Robust Local Non-parametric Conditional Estimation
• Shift Invariance Can Reduce Adversarial Robustness
• Towards Robust Neural Networks via Close-loop Control
• A Bayes-Optimal View on Adversarial Examples
• Benford's law: what does it say on adversarial images?
• On The Relationship Between Universal Adversarial Attacks And Sparse Representations
• Universal Approximation with Certified Networks
• Localized Uncertainty Attacks
• Understanding Robustness in Teacher-Student Setting: A New Perspective
• Adaptive versus Standard Descent Methods and Robustness Against Adversarial Examples