SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluation

Abstract

A black-box spectral method is introduced for evaluating the adversarial robustness of a given machine learning (ML) model. Our approach, named SPADE, exploits bijective distance mapping between the input/output graphs constructed for approximating the manifolds corresponding to the input/output data. By leveraging the generalized Courant-Fischer theorem, we propose a SPADE score for evaluating the adversarial robustness of a given model, which is proved to be an upper bound of the best Lipschitz constant under the manifold setting. To reveal the most non-robust data samples highly vulnerable to adversarial attacks, we develop a spectral graph embedding procedure leveraging dominant generalized eigenvectors. This embedding step allows assigning each data sample a robustness score that can be further harnessed for more effective adversarial training. Our experiments show the proposed SPADE method leads to promising empirical results for neural network models that are adversarially trained with the MNIST and CIFAR-10 data sets.

arXiv commentThe 2021 International Conference on Machine Learning (ICML)

References in corpus (11)

• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Fast is better than free: Revisiting adversarial training
• Security Evaluation of Pattern Classifiers under Attack
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Adversarial Machine Learning at Scale
• Detecting Adversarial Samples from Artifacts
• Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
• Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network
• Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets