Exploring the Space of Black-box Attacks on Deep Neural Networks

Abstract

Existing black-box attacks on deep neural networks (DNNs) so far have largely focused on transferability, where an adversarial instance generated for a locally trained model can "transfer" to attack other learning models. In this paper, we propose novel Gradient Estimation black-box attacks for adversaries with query access to the target model's class probabilities, which do not rely on transferability. We also propose strategies to decouple the number of queries required to generate each adversarial sample from the dimensionality of the input. An iterative variant of our attack achieves close to 100% adversarial success rates for both targeted and untargeted attacks on DNNs. We carry out extensive experiments for a thorough comparative evaluation of black-box attacks and show that the proposed Gradient Estimation attacks outperform all transferability based black-box attacks we tested on both MNIST and CIFAR-10 datasets, achieving adversarial success rates similar to well known, state-of-the-art white-box attacks. We also apply the Gradient Estimation attacks successfully against a real-world Content Moderation classifier hosted by Clarifai. Furthermore, we evaluate black-box attacks against state-of-the-art defenses. We show that the Gradient Estimation attacks are very effective even against these defenses.

arXiv comment25 pages, 7 figures, 10 tables

References in corpus (11)

• A Tutorial on Principal Component Analysis
• Wide Residual Networks
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• The Space of Transferable Adversarial Examples
• Simple Black-Box Adversarial Perturbations for Deep Networks
• DeepFool: a simple and accurate method to fool deep neural networks
• Fast Feature Fool: A data independent approach to universal adversarial perturbations
• Universal adversarial perturbations

Cited by in corpus (31)

• Adversarial Attack and Defense on Graph Data: A Survey
• Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors
• DARTS: Deceiving Autonomous Cars with Toxic Signs
• Why do deep convolutional networks generalize so poorly to small image transformations?
• Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach
• Adversarial Training against Location-Optimized Adversarial Patches
• Characterizing Attacks on Deep Reinforcement Learning
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• Hermes Attack: Steal DNN Models with Lossless Inference Accuracy
• Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
• Pixle: a fast and effective black-box attack based on rearranging pixels
• Paracosm: A Language and Tool for Testing Autonomous Driving Systems
• Fooling a Real Car with Adversarial Traffic Signs
• Hybrid Batch Attacks: Finding Black-box Adversarial Examples with Limited Queries
• Scratch that! An Evolution-based Adversarial Attack against Neural Networks
• On the Design of Black-box Adversarial Examples by Leveraging Gradient-free Optimization and Operator Splitting Method
• Stochastic Zeroth-order Optimization via Variance Reduction method
• There are No Bit Parts for Sign Bits in Black-Box Attacks
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• Nonlinear Projection Based Gradient Estimation for Query Efficient Blackbox Attacks
• Headless Horseman: Adversarial Attacks on Transfer Learning Models
• Effects of Loss Functions And Target Representations on Adversarial Robustness
• A Method for Computing Class-wise Universal Adversarial Perturbations
• Taking Care of The Discretization Problem: A Comprehensive Study of the Discretization Problem and A Black-Box Adversarial Attack in Discrete Integer Domain
• Projection & Probability-Driven Black-Box Attack
• A Frank-Wolfe Framework for Efficient and Effective Adversarial Attacks
• Accelerating Robustness Verification of Deep Neural Networks Guided by Target Labels
• Perception-in-the-Loop Adversarial Examples
• Universal Decision-Based Black-Box Perturbations: Breaking Security-Through-Obscurity Defenses
• Understanding Robustness in Teacher-Student Setting: A New Perspective
• Defending Against Adversarial Attacks Using Random Forests