Scratch that! An Evolution-based Adversarial Attack against Neural Networks

Abstract

We study black-box adversarial attacks for image classifiers in a constrained threat model, where adversaries can only modify a small fraction of pixels in the form of scratches on an image. We show that it is possible for adversaries to generate localized \textit{adversarial scratches} that cover less than $5\%$ of the pixels in an image and achieve targeted success rates of $98.77\%$ and $97.20\%$ on ImageNet and CIFAR-10 trained ResNet-50 models, respectively. We demonstrate that our scratches are effective under diverse shapes, such as straight lines or parabolic B\a'ezier curves, with single or multiple colors. In an extreme condition, in which our scratches are a single color, we obtain a targeted attack success rate of $66\%$ on CIFAR-10 with an order of magnitude fewer queries than comparable attacks. We successfully launch our attack against Microsoft's Cognitive Services Image Captioning API and propose various mitigation strategies.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Poisoning Attacks against Support Vector Machines
• Exploring the Space of Black-box Attacks on Deep Neural Networks
• Adversarial camera stickers: A physical camera-based attack on deep learning systems
• They Might NOT Be Giants: Crafting Black-Box Adversarial Examples with Fewer Queries Using Particle Swarm Optimization

Cited by in corpus (2)

• Pixle: a fast and effective black-box attack based on rearranging pixels
• A Singular Value Perspective on Model Robustness