Black-box Adversarial Attacks with Limited Queries and Information

Abstract

Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model. In practice, the threat model for real-world systems is often more restrictive than the typical black-box model where the adversary can observe the full output of the network on arbitrarily many chosen inputs. We define three realistic threat models that more accurately characterize many real-world classifiers: the query-limited setting, the partial-information setting, and the label-only setting. We develop new attacks that fool classifiers under these more restrictive threat models, where previous methods would be impractical or ineffective. We demonstrate that our methods are effective against an ImageNet classifier under our proposed threat models. We also demonstrate a targeted black-box attack against a commercial classifier, overcoming the challenges of limited query access, partial information, and other practical issues to break the Google Cloud Vision API.

arXiv commentICML 2018. This supercedes the previous paper "Query-efficient Black-box adversarial examples."

References in corpus (11)

• Intriguing properties of neural networks
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Poisoning Attacks against Support Vector Machines
• Synthesizing Robust Adversarial Examples
• Rethinking the Inception Architecture for Computer Vision
• Simple Black-Box Adversarial Perturbations for Deep Networks
• Approximation with Random Bases: Pro et Contra
• Black-Box Attacks against RNN based Malware Detection Algorithms

Cited by in corpus (106)

• Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspective
• A Review of Adversarial Attack and Defense for Classification Methods
• Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network
• Recent advances for quantum classifiers
• POBA-GA: Perturbation Optimized Black-Box Adversarial Attacks via Genetic Algorithm
• Semantic Robustness of Models of Source Code
• HopSkipJumpAttack: A Query-Efficient Decision-Based Attack
• Attacking Graph Convolutional Networks via Rewiring
• Characterizing and evaluating adversarial examples for Offline Handwritten Signature Verification
• Security Analysis of Deep Neural Networks Operating in the Presence of Cache Side-Channel Attacks
• WAF-A-MoLE: Evading Web Application Firewalls through Adversarial Machine Learning
• A survey of algorithmic recourse: definitions, formulations, solutions, and prospects
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
• Defending Against Universal Attacks Through Selective Feature Regeneration
• Gradient Starvation: A Learning Proclivity in Neural Networks
• Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
• AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks
• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• Robustness of Bayesian Neural Networks to Gradient-Based Attacks
• GenAttack: Practical Black-box Attacks with Gradient-Free Optimization
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• Investigating the significance of adversarial attacks and their relation to interpretability for radar-based human activity recognition systems
• A Survey of Black-Box Adversarial Attacks on Computer Vision Models
• Hiding Faces in Plain Sight: Disrupting AI Face Synthesis with Adversarial Perturbations
• GraphDefense: Towards Robust Graph Convolutional Networks
• Towards resilient machine learning for ransomware detection
• Black-box adversarial attacks using Evolution Strategies
• Yet another but more efficient black-box adversarial attack: tiling and evolution strategies
• Imitation Attacks and Defenses for Black-box Machine Translation Systems
• A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning
• Min-Max Optimization without Gradients: Convergence and Applications to Adversarial ML
• Scratch that! An Evolution-based Adversarial Attack against Neural Networks
• Security and Privacy for Artificial Intelligence: Opportunities and Challenges
• PatchAttack: A Black-box Texture-based Attack with Reinforcement Learning
• Local Black-box Adversarial Attacks: A Query Efficient Approach
• Enhancing Cross-task Transferability of Adversarial Examples with Dispersion Reduction
• Heuristic Black-box Adversarial Attacks on Video Recognition Models
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• DSRNA: Differentiable Search of Robust Neural Architectures
• Black-box Adversarial Attacks on Video Recognition Models
• Enhancing Cross-task Black-Box Transferability of Adversarial Examples with Dispersion Reduction
• Adversarial Robustness by Design through Analog Computing and Synthetic Gradients
• Mathematical Analysis of Adversarial Attacks
• Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluation
• Decision-based Universal Adversarial Attack
• Boosting Adversarial Attacks on Neural Networks with Better Optimizer
• GeoDA: a geometric framework for black-box adversarial attacks
• Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
• DAmageNet: A Universal Adversarial Dataset
• Do Fine-tuned Commonsense Language Models Really Generalize?
• An Efficient Adversarial Attack for Tree Ensembles
• Staircase Sign Method for Boosting Adversarial Attacks
• Adversarial Diffusion Attacks on Graph-based Traffic Prediction Models
• Adversarial Transferability in Wearable Sensor Systems
• Robustness Certificates Against Adversarial Examples for ReLU Networks
• Dealing with Adversarial Player Strategies in the Neural Network Game iNNk through Ensemble Learning
• On Adversarial Robustness of 3D Point Cloud Classification under Adaptive Attacks
• ZARTS: On Zero-order Optimization for Neural Architecture Search
• A Method for Computing Class-wise Universal Adversarial Perturbations
• Zeroth-Order Hybrid Gradient Descent: Towards A Principled Black-Box Optimization Framework
• Adversarial Examples in Constrained Domains
• Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation from a Blackbox Model
• Beware the Black-Box: on the Robustness of Recent Defenses to Adversarial Examples
• Identifying Model Weakness with Adversarial Examiner
• Is Face Recognition Safe from Realizable Attacks?
• Convergence of adaptive algorithms for weakly convex constrained optimization
• QEBA: Query-Efficient Boundary-Based Blackbox Attack
• RamBoAttack: A Robust Query Efficient Deep Neural Network Decision Exploit
• Delving into Data: Effectively Substitute Training for Black-box Attack
• RAIN: A Simple Approach for Robust and Accurate Image Classification Networks
• Gradient-Free Adversarial Attacks for Bayesian Neural Networks
• Towards Query Efficient Black-box Attacks: An Input-free Perspective
• CCA: Exploring the Possibility of Contextual Camouflage Attack on Object Detection
• Residual Error: a New Performance Measure for Adversarial Robustness
• Adversarial collision attacks on image hashing functions
• Towards Deep Learning Models Resistant to Large Perturbations
• An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness
• Diminishing the Effect of Adversarial Perturbations via Refining Feature Representation
• Robust Tracking against Adversarial Attacks
• Techniques for Adversarial Examples Threatening the Safety of Artificial Intelligence Based Systems
• Assessing the Adversarial Robustness of Monte Carlo and Distillation Methods for Deep Bayesian Neural Network Classification
• Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent
• Query complexity of adversarial attacks
• Feature Space Targeted Attacks by Statistic Alignment
• The Vulnerability of the Neural Networks Against Adversarial Examples in Deep Learning Algorithms
• Adversarial Attacks on Binary Image Recognition Systems
• The Effects of Image Distribution and Task on Adversarial Robustness
• Towards Imperceptible Query-limited Adversarial Attacks with Perceptual Feature Fidelity Loss
• Orthogonal Deep Models As Defense Against Black-Box Attacks
• Do Deep Minds Think Alike? Selective Adversarial Attacks for Fine-Grained Manipulation of Multiple Deep Neural Networks
• Adversarial Machine Learning for Cybersecurity and Computer Vision: Current Developments and Challenges
• Enhancing Transformation-based Defenses using a Distribution Classifier
• Thwarting finite difference adversarial attacks with output randomization
• Adversarial Eigen Attack on Black-Box Models
• LSDAT: Low-Rank and Sparse Decomposition for Decision-based Adversarial Attack
• An Efficient and Margin-Approaching Zero-Confidence Adversarial Attack
• BOSH: An Efficient Meta Algorithm for Decision-based Attacks
• Query-Free Adversarial Transfer via Undertrained Surrogates
• Hard-label Manifolds: Unexpected Advantages of Query Efficiency for Finding On-manifold Adversarial Examples
• A note on hyperparameters in black-box adversarial examples
• Adversarial Evaluation of Multimodal Models under Realistic Gray Box Assumption
• Self-Gradient Networks
• Adversarial Attacks for Multi-view Deep Models
• Learning to Disentangle Robust and Vulnerable Features for Adversarial Detection
• Universal Physical Camouflage Attacks on Object Detectors