Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks

Abstract

Adversarial attacks based on randomized search schemes have obtained state-of-the-art results in black-box robustness evaluation recently. However, as we demonstrate in this work, their efficiency in different query budget regimes depends on manual design and heuristic tuning of the underlying proposal distributions. We study how this issue can be addressed by adapting the proposal distribution online based on the information obtained during the attack. We consider Square Attack, which is a state-of-the-art score-based black-box attack, and demonstrate how its performance can be improved by a learned controller that adjusts the parameters of the proposal distribution online during the attack. We train the controller using gradient-based end-to-end training on a CIFAR10 model with white box access. We demonstrate that plugging the learned controller into the attack consistently improves its black-box robustness estimate in different query regimes by up to 20% for a wide range of different models with black-box access. We further show that the learned adaptation principle transfers well to the other data distributions such as CIFAR100 or ImageNet and to the targeted attack setting.

arXiv commentaccepted at NeurIPS 2021; updated the numbers in Table 5 and added references; added acknowledgements

References in corpus (27)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Theoretically Principled Trade-off between Robustness and Accuracy
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Fast is better than free: Revisiting adversarial training
• Adversarial Attacks and Defences: A Survey
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Learning to Learn without Gradient Descent by Gradient Descent
• Improving Black-box Adversarial Attacks with a Transfer-based Prior
• Simple Black-box Adversarial Attacks
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
• RobustBench: a standardized adversarial robustness benchmark
• MMA Training: Direct Input Space Margin Maximization through Adversarial Training
• Learning to Optimize: A Primer and A Benchmark
• Parsimonious Black-Box Adversarial Attacks via Efficient Combinatorial Optimization
• Query-efficient Meta Attack to Deep Neural Networks
• Black-box Adversarial Attacks with Bayesian Optimization
• Yet another but more efficient black-box adversarial attack: tiling and evolution strategies
• There are No Bit Parts for Sign Bits in Black-Box Attacks
• CorrAttack: Black-box Adversarial Attack with Structured Search
• Learning to Learn by Zeroth-Order Oracle
• Automated Discovery of Adaptive Attacks on Adversarial Defenses
• Meta Adversarial Training against Universal Patches