Adversarial Examples - A Complete Characterisation of the Phenomenon

Abstract

We provide a complete characterisation of the phenomenon of adversarial examples - inputs intentionally crafted to fool machine learning models. We aim to cover all the important concerns in this field of study: (1) the conjectures on the existence of adversarial examples, (2) the security, safety and robustness implications, (3) the methods used to generate and (4) protect against adversarial examples and (5) the ability of adversarial examples to transfer between different machine learning models. We provide ample background information in an effort to make this document self-contained. Therefore, this document can be used as survey, tutorial or as a catalog of attacks and defences using adversarial examples.

References in corpus (75)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Auto-Encoding Variational Bayes
• Distilling the Knowledge in a Neural Network
• Sequence to Sequence Learning with Neural Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• On the difficulty of training Recurrent Neural Networks
• Ensemble Adversarial Training: Attacks and Defenses
• A Tutorial on Principal Component Analysis
• Conditional Image Generation with PixelCNN Decoders
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Wide Residual Networks
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Tutorial on Variational Autoencoders
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Poisoning Attacks against Support Vector Machines
• Adversarial Feature Learning
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Adversarially Learned Inference
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Certified Defenses against Adversarial Examples
• Group Equivariant Convolutional Networks
• Adversarially Robust Generalization Requires More Data
• Robust Physical-World Attacks on Deep Learning Models
• Achieving Human Parity in Conversational Speech Recognition
• Generating Natural Adversarial Examples
• The Space of Transferable Adversarial Examples
• Security Evaluation of Pattern Classifiers under Attack
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Detecting Adversarial Samples from Artifacts
• Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN
• On the (Statistical) Detection of Adversarial Examples
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Black-box Adversarial Attacks with Limited Queries and Information
• Adversarial Perturbations Against Deep Neural Networks for Malware Classification
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
• Learning with a Strong Adversary
• A study of the effect of JPG compression on adversarial images
• Spatially Transformed Adversarial Examples
• Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression
• Adversarial Transformation Networks: Learning to Generate Adversarial Examples
• Stochastic Activation Pruning for Robust Adversarial Defense
• Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
• Motivating the Rules of the Game for Adversarial Example Research
• Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
• Adversarial vulnerability for any classifier
• MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples
• Foveation-based Mechanisms Alleviate Adversarial Examples
• A Boundary Tilting Persepective on the Phenomenon of Adversarial Examples
• Early Methods for Detecting Adversarial Images
• Robustness of classifiers: from adversarial to random noise
• Biologically inspired protection of deep networks from adversarial attacks
• Tactics of Adversarial Attack on Deep Reinforcement Learning Agents
• Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN
• Attacking the Madry Defense Model with $L_1$-based Adversarial Examples
• DeepCloak: Masking Deep Neural Network Models for Robustness Against Adversarial Samples
• Exploring the Space of Black-box Attacks on Deep Neural Networks
• Are Accuracy and Robustness Correlated?
• Standard detectors aren't (currently) fooled by physical adversarial stop signs
• Defending against Adversarial Images using Basis Functions Transformations
• Robustness of classifiers to universal perturbations: a geometric perspective
• Fortified Networks: Improving the Robustness of Deep Networks by Modeling the Manifold of Hidden Representations
• Defending Against Adversarial Attacks by Leveraging an Entire GAN
• Saturating Auto-Encoders
• Combating Adversarial Attacks Using Sparse Representations
• PAC-learning in the presence of evasion adversaries
• Robustness of Rotation-Equivariant Networks to Adversarial Perturbations
• Adversarially Robust Training through Structured Gradient Regularization
• HyperNetworks with statistical filtering for defending adversarial examples
• Featurized Bidirectional GAN: Adversarial Defense via Adversarially Learned Semantic Inference
• Blind Pre-Processing: A Robust Defense Method Against Adversarial Examples

Cited by in corpus (6)

• Robust Android Malware Detection System against Adversarial Attacks using Q-Learning
• Arms Race in Adversarial Malware Detection: A Survey
• Towards Robust and Stable Deep Learning Algorithms for Forward Backward Stochastic Differential Equations
• Active Subspace of Neural Networks: Structural Analysis and Universal Attacks
• Neural Network Verification for the Masses (of AI graduates)
• Linking average- and worst-case perturbation robustness via class selectivity and dimensionality