Hessian-Aware Zeroth-Order Optimization for Black-Box Adversarial Attack

Abstract

Zeroth-order optimization is an important research topic in machine learning. In recent years, it has become a key tool in black-box adversarial attack to neural network based image classifiers. However, existing zeroth-order optimization algorithms rarely extract second-order information of the model function. In this paper, we utilize the second-order information of the objective function and propose a novel \textit{Hessian-aware zeroth-order algorithm} called \texttt{ZO-HessAware}. Our theoretical result shows that \texttt{ZO-HessAware} has an improved zeroth-order convergence rate and query complexity under structured Hessian approximation, where we propose a few approximation methods for estimating Hessian. Our empirical studies on the black-box adversarial attack problem validate that our algorithm can achieve improved success rates with a lower query complexity.

References in corpus (8)

• Practical Bayesian Optimization of Machine Learning Algorithms
• ADADELTA: An Adaptive Learning Rate Method
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN
• SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator
• An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedback
• Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization

Cited by in corpus (10)

• A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning
• Zeroth-Order Algorithms for Smooth Saddle-Point Problems
• ZO-JADE: Zeroth-order Curvature-Aware Multi-Agent Convex Optimization
• Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra
• Hessian Inverse Approximation as Covariance for Random Perturbation in Black-Box Problems
• Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent
• AdvMind: Inferring Adversary Intent of Black-Box Attacks
• Learning Sampling Policy for Faster Derivative Free Optimization
• On the Second-order Convergence Properties of Random Search Methods
• Mirror Natural Evolution Strategies