A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization
arXiv:2102.10707
Abstract
We consider the zeroth-order optimization problem in the huge-scale setting, where the dimension of the problem is so large that performing even basic vector operations on the decision variables is infeasible. In this paper, we propose a novel algorithm, coined ZO-BCD, that exhibits favorable overall query complexity and has a much smaller per-iteration computational complexity. In addition, we discuss how the memory footprint of ZO-BCD can be reduced even further by the clever use of circulant measurement matrices. As an application of our new method, we propose the idea of crafting adversarial attacks on neural network based classifiers in a wavelet domain, which can result in problem dimensions of over 1.7 million. In particular, we show that crafting adversarial examples to audio classifiers in a wavelet domain can achieve the state-of-the-art attack success rate of 97.9%.
arXiv commentAccepted to ICML 2021
References in corpus (12)
- ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
- Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Did you hear that? Adversarial Examples Against Automatic Speech Recognition
- A Comprehensive Linear Speedup Analysis for Asynchronous Stochastic Parallel Optimization from Zeroth-Order to First-Order
- Stochastic Zeroth-order Optimization in High Dimensions
- Gradientless Descent: High-Dimensional Zeroth-Order Optimization
- ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization
- Zeroth-Order Regularized Optimization (ZORO): Approximately Sparse Gradients and Adaptive Sampling
- Universal adversarial examples in speech command classification
- Enabling Fast and Universal Audio Adversarial Attack Using Generative Model
- Improved bounds for the RIP of Subsampled Circulant matrices