Delving into Transferable Adversarial Examples and Black-box Attacks

Abstract

An intriguing property of deep neural networks is the existence of adversarial examples, which can transfer among different architectures. These transferable adversarial examples may severely hinder deep neural network-based applications. Previous works mostly study the transferability using small scale datasets. In this work, we are the first to conduct an extensive study of the transferability over large models and a large scale dataset, and we are also the first to study the transferability of targeted adversarial examples with their target labels. We study both non-targeted and targeted adversarial examples, and show that while transferable non-targeted adversarial examples are easy to find, targeted adversarial examples generated using existing approaches almost never transfer with their target labels. Therefore, we propose novel ensemble-based approaches to generating transferable adversarial examples. Using such approaches, we observe a large proportion of targeted adversarial examples that are able to transfer with their target labels for the first time. We also present some geometric studies to help understanding the transferable adversarial examples. Finally, we show that the adversarial examples generated using ensemble-based approaches can successfully attack Clarifai.com, which is a black-box image classification system.

Cited by in corpus (302)

• Ensemble Adversarial Training: Attacks and Defenses
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Synthesizing Robust Adversarial Examples
• On Evaluating Adversarial Robustness
• Robust Physical-World Attacks on Deep Learning Models
• Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
• Synthetic and Natural Noise Both Break Neural Machine Translation
• Adversarial Examples Are Not Bugs, They Are Features
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Efficient Neural Network Robustness Certification with General Activation Functions
• Black-box Adversarial Attacks with Limited Queries and Information
• Adversarial Attack and Defense on Graph Data: A Survey
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Formal Security Analysis of Neural Networks using Symbolic Intervals
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
• Pathologies of Neural Models Make Interpretations Difficult
• Adversarial Examples: Opportunities and Challenges
• Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors
• Generating Adversarial Examples with Adversarial Networks
• Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks
• Adversarial Examples: Attacks and Defenses for Deep Learning
• MagNet: a Two-Pronged Defense against Adversarial Examples
• DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars
• Mitigating Adversarial Effects Through Randomization
• Evaluating and Improving Adversarial Robustness of Machine Learning-Based Network Intrusion Detectors
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• DARTS: Deceiving Autonomous Cars with Toxic Signs
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
• Are Labels Required for Improving Adversarial Robustness?
• Improving Black-box Adversarial Attacks with a Transfer-based Prior
• Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspective
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Understanding Membership Inferences on Well-Generalized Learning Models
• Quantum noise protects quantum classifiers against adversaries
• Towards Fast Computation of Certified Robustness for ReLU Networks
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• Adversarial Attack Vulnerability of Medical Image Analysis Systems: Unexplored Factors
• Constructing Unrestricted Adversarial Examples with Generative Models
• Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Blind Backdoors in Deep Learning Models
• A Review of Adversarial Attack and Defense for Classification Methods
• Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network
• Improving Adversarial Robustness of Ensembles with Diversity Training
• Universal Adversarial Examples in Remote Sensing: Methodology and Benchmark
• Characterizing Audio Adversarial Examples Using Temporal Dependency
• AttriGuard: A Practical Defense Against Attribute Inference Attacks via Adversarial Machine Learning
• Understanding and Enhancing the Transferability of Adversarial Examples
• Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks
• Adversarial Examples for Semantic Segmentation and Object Detection
• Low Frequency Adversarial Perturbation
• Attacking the Madry Defense Model with $L_1$-based Adversarial Examples
• Towards Adversarial Malware Detection: Lessons Learned from PDF-based Attacks
• Improving the Generalization of Adversarial Training with Domain Adaptation
• A Closer Look at Accuracy vs. Robustness
• Improving Transferability of Adversarial Examples with Input Diversity
• Adversarial Attack and Defense on Point Sets
• Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems
• HopSkipJumpAttack: A Query-Efficient Decision-Based Attack
• Invisible Mask: Practical Attacks on Face Recognition with Infrared
• Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness
• Privacy Intelligence: A Survey on Image Privacy in Online Social Networks
• EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
• Improving the Transferability of Adversarial Examples with Resized-Diverse-Inputs, Diversity-Ensemble and Region Fitting
• Robust Local Features for Improving the Generalization of Adversarial Training
• A Survey of Game Theoretic Approaches for Adversarial Machine Learning in Cybersecurity Tasks
• Stealing Hyperparameters in Machine Learning
• RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
• Boosting Adversarial Attacks with Momentum
• Adversarial Examples in Modern Machine Learning: A Review
• Fawkes: Protecting Privacy against Unauthorized Deep Learning Models
• On Success and Simplicity: A Second Look at Transferable Targeted Attacks
• Diversity can be Transferred: Output Diversification for White- and Black-box Attacks
• Defending against Adversarial Images using Basis Functions Transformations
• NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing
• A black-box adversarial attack for poisoning clustering
• Learning Black-Box Attackers with Transferable Priors and Query Feedback
• SafetyNet: Detecting and Rejecting Adversarial Examples Robustly
• Robustness Verification of Tree-based Models
• Enhancing Adversarial Example Transferability with an Intermediate Level Attack
• Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching
• Fooling OCR Systems with Adversarial Text Images
• Universal Adversarial Audio Perturbations
• Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
• Mimic and Fool: A Task Agnostic Adversarial Attack
• Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions
• Adversarial Attacks for Optical Flow-Based Action Recognition Classifiers
• Bypassing Feature Squeezing by Increasing Adversary Strength
• AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks
• Curriculum Adversarial Training
• Improving the Transferability of Adversarial Examples with Arbitrary Style Transfer
• Perturbing Across the Feature Hierarchy to Improve Standard and Strict Blackbox Attack Transferability
• Learning to Defend by Learning to Attack
• Attention Meets Perturbations: Robust and Interpretable Attention with Adversarial Training
• On the Limitation of Local Intrinsic Dimensionality for Characterizing the Subspaces of Adversarial Examples
• Learning Adversary-Resistant Deep Neural Networks
• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• Divide, Denoise, and Defend against Adversarial Attacks
• Adversarial Examples in Deep Learning: Characterization and Divergence
• GenAttack: Practical Black-box Attacks with Gradient-Free Optimization
• Task dependent Deep LDA pruning of neural networks
• Query-Efficient Black-box Adversarial Examples (superceded)
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• Defense against Universal Adversarial Perturbations
• Two Sides of the Same Coin: White-box and Black-box Attacks for Transfer Learning
• ECGadv: Generating Adversarial Electrocardiogram to Misguide Arrhythmia Classification System
• Pick-Object-Attack: Type-Specific Adversarial Attack for Object Detection
• Deep Neural Network Fingerprinting by Conferrable Adversarial Examples
• Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective
• Defense Against Adversarial Attacks with Saak Transform
• Adversarial Reprogramming of Neural Networks
• Transferable Perturbations of Deep Feature Distributions
• HashTran-DNN: A Framework for Enhancing Robustness of Deep Neural Networks against Adversarial Malware Samples
• Imitation Attacks and Defenses for Black-box Machine Translation Systems
• Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning
• Hybrid Batch Attacks: Finding Black-box Adversarial Examples with Limited Queries
• A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inference
• Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses
• Min-Max Optimization without Gradients: Convergence and Applications to Adversarial ML
• DUP-Net: Denoiser and Upsampler Network for 3D Adversarial Point Clouds Defense
• Interpretable Deep Learning under Fire
• PAC-learning in the presence of evasion adversaries
• Zeroth-Order Algorithms for Nonconvex Minimax Problems with Improved Complexities
• MULDEF: Multi-model-based Defense Against Adversarial Examples for Neural Networks
• Efficient Decision-based Black-box Adversarial Attacks on Face Recognition
• Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization
• A Unified Approach to Interpreting and Boosting Adversarial Transferability
• ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies
• Android HIV: A Study of Repackaging Malware for Evading Machine-Learning Detection
• Membership Leakage in Label-Only Exposures
• Adversarially Robust Training through Structured Gradient Regularization
• Sparse Adversarial Perturbations for Videos
• ML-LOO: Detecting Adversarial Examples with Feature Attribution
• On the Design of Black-box Adversarial Examples by Leveraging Gradient-free Optimization and Operator Splitting Method
• Attacking Graph-based Classification via Manipulating the Graph Structure
• Physical Adversarial Attacks on Deep Neural Networks for Traffic Sign Recognition: A Feasibility Study
• Curls & Whey: Boosting Black-Box Adversarial Attacks
• Adversarial Attacks on Brain-Inspired Hyperdimensional Computing-Based Classifiers
• An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models
• Robust Multilingual Part-of-Speech Tagging via Adversarial Training
• Learning Transferable Adversarial Examples via Ghost Networks
• Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognition
• Learning deep forest with multi-scale Local Binary Pattern features for face anti-spoofing
• Gradient Band-based Adversarial Training for Generalized Attack Immunity of A3C Path Finding
• Generic Black-Box End-to-End Attack Against State of the Art API Call Based Malware Classifiers
• PatchAttack: A Black-box Texture-based Attack with Reinforcement Learning
• Towards Robust Detection of Adversarial Examples
• Adversarial Example Games
• Adversarial Image Color Transformations in Explicit Color Filter Space
• Interpretability is a Kind of Safety: An Interpreter-based Ensemble for Adversary Defense
• On the Limitation of MagNet Defense against $L_1$-based Adversarial Examples
• Subverting Fair Image Search with Generative Adversarial Perturbations
• Post-breach Recovery: Protection against White-box Adversarial Examples for Leaked DNN Models
• Bridging the Performance Gap between FGSM and PGD Adversarial Training
• Generating Semantic Adversarial Examples via Feature Manipulation
• Adversarial Attacks Beyond the Image Space
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Snooping Attacks on Deep Reinforcement Learning
• Sitatapatra: Blocking the Transfer of Adversarial Samples
• On the Limitations of Denoising Strategies as Adversarial Defenses
• EREBA: Black-box Energy Testing of Adaptive Neural Networks
• Query Attack via Opposite-Direction Feature:Towards Robust Image Retrieval
• Towards Practical Lottery Ticket Hypothesis for Adversarial Training
• Fooling Vision and Language Models Despite Localization and Attention Mechanism
• GhostImage: Remote Perception Attacks against Camera-based Image Classification Systems
• Adaptive Generation of Unrestricted Adversarial Inputs
• Adversarial Metric Attack and Defense for Person Re-identification
• Sparse Adversarial Video Attacks with Spatial Transformations
• Boosting Adversarial Attacks on Neural Networks with Better Optimizer
• White-to-Black: Efficient Distillation of Black-Box Adversarial Attacks
• Decision-based Universal Adversarial Attack
• Bit Error Robustness for Energy-Efficient DNN Accelerators
• Nonlinear Projection Based Gradient Estimation for Query Efficient Blackbox Attacks
• Ulixes: Facial Recognition Privacy with Adversarial Machine Learning
• Protecting Against Image Translation Deepfakes by Leaking Universal Perturbations from Black-Box Neural Networks
• Improving the Transferability of Adversarial Attacks on Face Recognition with Beneficial Perturbation Feature Augmentation
• Boosting Black-Box Attack with Partially Transferred Conditional Adversarial Distribution
• Towards Robust Training of Neural Networks by Regularizing Adversarial Gradients
• Mathematical Analysis of Adversarial Attacks
• Universal, transferable and targeted adversarial attacks
• Detecting Adversarial Examples via Key-based Network
• Targeted Nonlinear Adversarial Perturbations in Images and Videos
• Generating Universal Adversarial Perturbations for Quantum Classifiers
• Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification
• Regula Sub-rosa: Latent Backdoor Attacks on Deep Neural Networks
• Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models
• TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness
• How Wrong Am I? - Studying Adversarial Examples and their Impact on Uncertainty in Gaussian Process Machine Learning Models
• Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness
• Identify Susceptible Locations in Medical Records via Adversarial Attacks on Deep Predictive Models
• Patch-wise++ Perturbation for Adversarial Targeted Attacks
• Defending against Machine Learning based Inference Attacks via Adversarial Examples: Opportunities and Challenges
• Improving Certified Robustness via Statistical Learning with Logical Reasoning
• CAP-GAN: Towards Adversarial Robustness with Cycle-consistent Attentional Purification
• Robust Attacks against Multiple Classifiers
• Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
• A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples
• Adversarial Detection Avoidance Attacks: Evaluating the robustness of perceptual hashing-based client-side scanning
• Defending against Adversarial Attack towards Deep Neural Networks via Collaborative Multi-task Training
• Adversarial Defense by Stratified Convolutional Sparse Coding
• Exploring the Back Alleys: Analysing The Robustness of Alternative Neural Network Architectures against Adversarial Attacks
• Improving Adversarial Transferability with Gradient Refining
• $n$-ML: Mitigating Adversarial Examples via Ensembles of Topologically Manipulated Classifiers
• Stochastic Substitute Training: A Gray-box Approach to Craft Adversarial Examples Against Gradient Obfuscation Defenses
• The Efficacy of SHIELD under Different Threat Models
• Regularizers for Single-step Adversarial Training
• On the Robustness of Human Pose Estimation
• Who's Afraid of Adversarial Queries? The Impact of Image Modifications on Content-based Image Retrieval
• The Adversarial Attack and Detection under the Fisher Information Metric
• Fast Gradient Non-sign Methods
• Towards Evaluating the Robustness of Neural Networks Learned by Transduction
• CAAD 2018: Generating Transferable Adversarial Examples
• One Bit Matters: Understanding Adversarial Examples as the Abuse of Redundancy
• RamBoAttack: A Robust Query Efficient Deep Neural Network Decision Exploit
• Stochastic Combinatorial Ensembles for Defending Against Adversarial Examples
• BUZz: BUffer Zones for defending adversarial examples in image classification
• Spanning Attack: Reinforce Black-box Attacks with Unlabeled Data
• SEALing Neural Network Models in Secure Deep Learning Accelerators
• Taking Care of The Discretization Problem: A Comprehensive Study of the Discretization Problem and A Black-Box Adversarial Attack in Discrete Integer Domain
• Cloud-based Image Classification Service Is Not Robust To Simple Transformations: A Forgotten Battlefield
• Switching Transferable Gradient Directions for Query-Efficient Black-Box Adversarial Attacks
• PhysGAN: Generating Physical-World-Resilient Adversarial Examples for Autonomous Driving
• Beware the Black-Box: on the Robustness of Recent Defenses to Adversarial Examples
• Prior Networks for Detection of Adversarial Attacks
• Towards Query Efficient Black-box Attacks: An Input-free Perspective
• A Frank-Wolfe Framework for Efficient and Effective Adversarial Attacks
• Patch-wise Attack for Fooling Deep Neural Network
• Adversarial Example Detection by Classification for Deep Speech Recognition
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• A Cyclically-Trained Adversarial Network for Invariant Representation Learning
• A New Ensemble Method for Concessively Targeted Multi-model Attack
• AdvFoolGen: Creating Persistent Troubles for Deep Classifiers
• Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability
• Adversarial Robustness Guarantees for Random Deep Neural Networks
• An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness
• Defective Convolutional Networks
• Smoothed Inference for Adversarially-Trained Models
• STA: Adversarial Attacks on Siamese Trackers
• Diminishing the Effect of Adversarial Perturbations via Refining Feature Representation
• A Unified Game-Theoretic Interpretation of Adversarial Robustness
• New CleverHans Feature: Better Adversarial Robustness Evaluations with Attack Bundling
• Implicit Generative Modeling of Random Noise during Training for Adversarial Robustness
• Multi-stage Optimization based Adversarial Training
• Towards Black-box Attacks on Deep Learning Apps
• On the Exploitability of Audio Machine Learning Pipelines to Surreptitious Adversarial Examples
• Exploring Transferable and Robust Adversarial Perturbation Generation from the Perspective of Network Hierarchy
• Avengers Ensemble! Improving Transferability of Authorship Obfuscation
• Anti-Bandit Neural Architecture Search for Model Defense
• Yet Another Intermediate-Level Attack
• Query-Free Attacks on Industry-Grade Face Recognition Systems under Resource Constraints
• Search Space of Adversarial Perturbations against Image Filters
• Regularized Ensembles and Transferability in Adversarial Learning
• Sample Complexity of Adversarially Robust Linear Classification on Separated Data
• On Adaptive Distance Estimation
• When are Non-Parametric Methods Robust?
• Simulating Unknown Target Models for Query-Efficient Black-box Attacks
• Colored Noise Injection for Training Adversarially Robust Neural Networks
• Who's Afraid of Adversarial Transferability?
• "What's in the box?!": Deflecting Adversarial Attacks by Randomly Deploying Adversarially-Disjoint Models
• Adversarial Robustness on Image Classification with $k$-means
• Technical Report: When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks
• Simple and Efficient Hard Label Black-box Adversarial Attacks in Low Query Budget Regimes
• Adversarial Learning with Cost-Sensitive Classes
• Query complexity of adversarial attacks
• Defending Adversarial Attacks via Semantic Feature Manipulation
• A Study on the Uncertainty of Convolutional Layers in Deep Neural Networks
• Cross-domain Cross-architecture Black-box Attacks on Fine-tuned Models with Transferred Evolutionary Strategies
• Renofeation: A Simple Transfer Learning Method for Improved Adversarial Robustness
• Adversarial Evaluation of Multimodal Models under Realistic Gray Box Assumption
• Generating Adversarial Perturbation with Root Mean Square Gradient
• Architectural Resilience to Foreground-and-Background Adversarial Noise
• On the Vulnerability of CNN Classifiers in EEG-Based BCIs
• Defense Through Diverse Directions
• Do Deep Minds Think Alike? Selective Adversarial Attacks for Fine-Grained Manipulation of Multiple Deep Neural Networks
• Adversarial Imitation Attack
• Extending Adversarial Attacks to Produce Adversarial Class Probability Distributions
• blessing in disguise: Designing Robust Turing Test by Employing Algorithm Unrobustness
• ROSA: Robust Salient Object Detection against Adversarial Attacks
• Defense-guided Transferable Adversarial Attacks
• Analyzing Adversarial Robustness of Deep Neural Networks in Pixel Space: a Semantic Perspective
• Adaptive Verifiable Training Using Pairwise Class Similarity
• Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses
• Visualizing Representations of Adversarially Perturbed Inputs
• Ask, Acquire, and Attack: Data-free UAP Generation using Class Impressions
• Improved Algorithms for White-Box Adversarial Streams
• Consistent Non-Parametric Methods for Maximizing Robustness
• Query-Free Adversarial Transfer via Undertrained Surrogates
• Multi-way Encoding for Robustness
• Exploring Adversarial Fake Images on Face Manifold
• advPattern: Physical-World Attacks on Deep Person Re-Identification via Adversarially Transformable Patterns
• Towards Model-Agnostic Adversarial Defenses using Adversarially Trained Autoencoders
• Using LIP to Gloss Over Faces in Single-Stage Face Detection Networks
• Learning to Disentangle Robust and Vulnerable Features for Adversarial Detection
• Localized Adversarial Training for Increased Accuracy and Robustness in Image Classification
• Beyond Universal Person Re-ID Attack
• Efficient and Transferable Adversarial Examples from Bayesian Neural Networks
• On the Transferability of Adversarial Attacksagainst Neural Text Classifier