Stateful Detection of Black-Box Adversarial Attacks

Abstract

The problem of adversarial examples, evasion attacks on machine learning classifiers, has proven extremely difficult to solve. This is true even when, as is the case in many practical settings, the classifier is hosted as a remote service and so the adversary does not have direct access to the model parameters. This paper argues that in such settings, defenders have a much larger space of actions than have been previously explored. Specifically, we deviate from the implicit assumption made by prior work that a defense must be a stateless function that operates on individual examples, and explore the possibility for stateful defenses. To begin, we develop a defense designed to detect the process of adversarial example generation. By keeping a history of the past queries, a defender can try to identify when a sequence of queries appears to be for the purpose of generating an adversarial example. We then introduce query blinding, a new class of attacks designed to bypass defenses that rely on such a defense approach. We believe that expanding the study of adversarial examples from stateless classifiers to stateful systems is not only more realistic for many black-box settings, but also gives the defender a much-needed advantage in responding to the adversary.

References in corpus (13)

• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• On Evaluating Adversarial Robustness
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Detecting Adversarial Samples from Artifacts
• On the (Statistical) Detection of Adversarial Examples
• On Detecting Adversarial Perturbations
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Guessing Smart: Biased Sampling for Efficient Black-Box Adversarial Attacks
• Evaluating and Understanding the Robustness of Adversarial Logit Pairing
• E-LPIPS: Robust Perceptual Image Similarity via Random Transformation Ensembles
• RED-Attack: Resource Efficient Decision based Attack for Machine Learning
• Sitatapatra: Blocking the Transfer of Adversarial Samples

Cited by in corpus (3)

• Hybrid Batch Attacks: Finding Black-box Adversarial Examples with Limited Queries
• Percival: Making In-Browser Perceptual Ad Blocking Practical With Deep Learning
• AdvMind: Inferring Adversary Intent of Black-Box Attacks