Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
arXiv:1905.01726
Abstract
A large body of recent work has investigated the phenomenon of evasion attacks using adversarial examples for deep learning systems, where the addition of norm-bounded perturbations to the test inputs leads to incorrect output classification. Previous work has investigated this phenomenon in closed-world systems where training and test inputs follow a pre-specified distribution. However, real-world implementations of deep learning applications, such as autonomous driving and content classification are likely to operate in the open-world environment. In this paper, we demonstrate the success of open-world evasion attacks, where adversarial examples are generated from out-of-distribution inputs (OOD adversarial examples). In our study, we use 11 state-of-the-art neural network models trained on 3 image datasets of varying complexity. We first demonstrate that state-of-the-art detectors for out-of-distribution data are not robust against OOD adversarial examples. We then consider 5 known defenses for adversarial examples, including state-of-the-art robust training methods, and show that against these defenses, OOD adversarial examples can achieve up to 4 higher target success rates compared to adversarial examples generated from in-distribution data. We also take a quantitative look at how open-world evasion attacks may affect real-world systems. Finally, we present the first steps towards a robust open-world machine learning system.
arXiv comment18 pages, 5 figures, 9 tables
References in corpus (51)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Natural Language Processing (almost) from Scratch
- Striving for Simplicity: The All Convolutional Net
- Densely Connected Convolutional Networks
- Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
- Wide Residual Networks
- ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
- Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
- Deep Learning for Anomaly Detection: A Survey
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Synthesizing Robust Adversarial Examples
- Certified Defenses against Adversarial Examples
- Countering Adversarial Images using Input Transformations
- Deep Anomaly Detection with Outlier Exposure
- Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
- Adversarial Machine Learning at Scale
- On the (Statistical) Detection of Adversarial Examples
- Learning Confidence for Out-of-Distribution Detection in Neural Networks
- Certifying Some Distributional Robustness with Principled Adversarial Training
- Black-box Adversarial Attacks with Limited Queries and Information
- PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
- Formal Security Analysis of Neural Networks using Symbolic Intervals
- On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
- Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
- To Trust Or Not To Trust A Classifier
- MagNet: a Two-Pronged Defense against Adversarial Examples
- ShakeDrop Regularization for Deep Residual Learning
- Anomaly Detection using One-Class Neural Networks
- Selective Classification for Deep Neural Networks
- Simple Black-Box Adversarial Perturbations for Deep Networks
- Houdini: Fooling Deep Structured Prediction Models
- Towards Fast Computation of Certified Robustness for ReLU Networks
- MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples
- Delving into adversarial attacks on deep policies
- Scaling provable adversarial defenses
- Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
- Adversarial Examples that Fool Detectors
- Evasion and Hardening of Tree Ensemble Classifiers
- Reducing Network Agnostophobia
- Evaluating and Understanding the Robustness of Adversarial Logit Pairing
- Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong
- CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition
- Adversarial Examples for Semantic Image Segmentation
- Open Category Detection with PAC Guarantees
- Adversarial examples for generative models
- Rogue Signs: Deceiving Traffic Sign Recognition with Malicious Ads and Logos
- Classification-Reconstruction Learning for Open-Set Recognition