Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks

Abstract

Adversarial attacks based on randomized search schemes have obtained state-of-the-art results in black-box robustness evaluation recently. However, as we demonstrate in this work, their efficiency in different query budget regimes depends on manual design and heuristic tuning of the underlying proposal distributions. We study how this issue can be addressed by adapting the proposal distribution online based on the information obtained during the attack. We consider Square Attack, which is a state-of-the-art score-based black-box attack, and demonstrate how its performance can be improved by a learned controller that adjusts the parameters of the proposal distribution online during the attack. We train the controller using gradient-based end-to-end training on a CIFAR10 model with white box access. We demonstrate that plugging the learned controller into the attack consistently improves its black-box robustness estimate in different query regimes by up to 20% for a wide range of different models with black-box access. We further show that the learned adaptation principle transfers well to the other data distributions such as CIFAR100 or ImageNet and to the targeted attack setting.

accepted at NeurIPS 2021; updated the numbers in Table 5 and added references; added acknowledgements

References in corpus (15)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Theoretically Principled Trade-off between Robustness and Accuracy
• Fast is better than free: Revisiting adversarial training
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
• RobustBench: a standardized adversarial robustness benchmark
• Learning to Optimize: A Primer and A Benchmark
• Black-box Adversarial Attacks with Bayesian Optimization
• Yet another but more efficient black-box adversarial attack: tiling and evolution strategies
• There are No Bit Parts for Sign Bits in Black-Box Attacks
• Learning to Learn by Zeroth-Order Oracle
• CorrAttack: Black-box Adversarial Attack with Structured Search
• Automated Discovery of Adaptive Attacks on Adversarial Defenses
• Meta Adversarial Training against Universal Patches