Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models

Abstract

Many machine learning algorithms are vulnerable to almost imperceptible perturbations of their inputs. So far it was unclear how much risk adversarial perturbations carry for the safety of real-world machine learning applications because most methods used to generate such perturbations rely either on detailed model information (gradient-based attacks) or on confidence scores such as class probabilities (score-based attacks), neither of which are available in most real-world scenarios. In many such cases one currently needs to retreat to transfer-based attacks which rely on cumbersome substitute models, need access to the training data and can be defended against. Here we emphasise the importance of attacks which solely rely on the final model decision. Such decision-based attacks are (1) applicable to real-world black-box models such as autonomous cars, (2) need less knowledge and are easier to apply than transfer-based attacks and (3) are more robust to simple defences than gradient- or score-based attacks. Previous attacks in this category were limited to simple models or simple datasets. Here we introduce the Boundary Attack, a decision-based attack that starts from a large adversarial perturbation and then seeks to reduce the perturbation while staying adversarial. The attack is conceptually simple, requires close to no hyperparameter tuning, does not rely on substitute models and is competitive with the best gradient-based attacks in standard computer vision tasks like ImageNet. We apply the attack on two black-box algorithms from Clarifai.com. The Boundary Attack in particular and the class of decision-based attacks in general open new avenues to study the robustness of machine learning models and raise new questions regarding the safety of deployed machine learning systems. An implementation of the attack is available as part of Foolbox at https://github.com/bethgelab/foolbox .

arXiv commentPublished as a conference paper at the Sixth International Conference on Learning Representations (ICLR 2018) https://openreview.net/forum?id=SyZI0GWCZ

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Intriguing properties of neural networks
• Ensemble Adversarial Training: Attacks and Defenses
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Defensive Distillation is Not Robust to Adversarial Examples
• Houdini: Fooling Deep Structured Prediction Models
• DeepFool: a simple and accurate method to fool deep neural networks
• SafetyNet: Detecting and Rejecting Adversarial Examples Robustly
• Comment on "Biologically inspired protection of deep networks from adversarial attacks"

Cited by in corpus (24)

• Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspective
• Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing
• advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
• Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks
• Adaptative Perturbation Patterns: Realistic Adversarial Learning for Robust Intrusion Detection
• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• Black-box adversarial attacks using Evolution Strategies
• Simple black-box universal adversarial attacks on medical image classification based on deep neural networks
• Training Robust Deep Models for Time-Series Domain: Novel Algorithms and Theoretical Analysis
• RED-Attack: Resource Efficient Decision based Attack for Machine Learning
• Nonlinear Projection Based Gradient Estimation for Query Efficient Blackbox Attacks
• Boosting Adversarial Attacks on Neural Networks with Better Optimizer
• Denoised Internal Models: a Brain-Inspired Autoencoder against Adversarial Attacks
• RamBoAttack: A Robust Query Efficient Deep Neural Network Decision Exploit
• QEBA: Query-Efficient Boundary-Based Blackbox Attack
• Beware the Black-Box: on the Robustness of Recent Defenses to Adversarial Examples
• Bandlimiting Neural Networks Against Adversarial Attacks
• TETRIS: Towards Exploring the Robustness of Interactive Segmentation
• Simple and Efficient Hard Label Black-box Adversarial Attacks in Low Query Budget Regimes
• The Human Visual System and Adversarial AI
• Bio-inspired Robustness: A Review
• Shape Defense Against Adversarial Attacks
• Sparse Coding Frontend for Robust Neural Networks
• Closing the Gap: Achieving Better Accuracy-Robustness Tradeoffs against Query-Based Attacks