Automated Discovery of Adaptive Attacks on Adversarial Defenses

Abstract

Reliable evaluation of adversarial defenses is a challenging task, currently limited to an expert who manually crafts attacks that exploit the defense's inner workings or approaches based on an ensemble of fixed attacks, none of which may be effective for the specific defense at hand. Our key observation is that adaptive attacks are composed of reusable building blocks that can be formalized in a search space and used to automatically discover attacks for unknown defenses. We evaluated our approach on 24 adversarial defenses and show that it outperforms AutoAttack, the current state-of-the-art tool for reliable evaluation of adversarial defenses: our tool discovered significantly stronger attacks by producing 3.0\%-50.8\% additional adversarial examples for 10 models, while obtaining attacks with slightly stronger or similar strength for the remaining models.

arXiv comment21 pages, 3 figures, 10 tables. NeurIPS2021

References in corpus (16)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Neural Architecture Search with Reinforcement Learning
• Neural Architecture Search: A Survey
• DARTS: Differentiable Architecture Search
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• On the (Statistical) Detection of Adversarial Examples
• Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression
• On Detecting Adversarial Perturbations
• Improving Adversarial Robustness via Promoting Ensemble Diversity
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• On Adaptive Attacks to Adversarial Example Defenses
• RobustBench: a standardized adversarial robustness benchmark
• A New Defense Against Adversarial Images: Turning a Weakness into a Strength
• Defending Against Machine Learning Model Stealing Attacks Using Deceptive Perturbations
• Adversarial Robustness Toolbox v1.0.0
• Composite Adversarial Attacks

Cited by in corpus (2)

• Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks
• Improving Fast Minimum-Norm Attacks with Hyperparameter Optimization