On the Limitation of Local Intrinsic Dimensionality for Characterizing the Subspaces of Adversarial Examples

Abstract

Understanding and characterizing the subspaces of adversarial examples aid in studying the robustness of deep neural networks (DNNs) to adversarial perturbations. Very recently, Ma et al. (ICLR 2018) proposed to use local intrinsic dimensionality (LID) in layer-wise hidden representations of DNNs to study adversarial subspaces. It was demonstrated that LID can be used to characterize the adversarial subspaces associated with different attack methods, e.g., the Carlini and Wagner's (C&W) attack and the fast gradient sign attack. In this paper, we use MNIST and CIFAR-10 to conduct two new sets of experiments that are absent in existing LID analysis and report the limitation of LID in characterizing the corresponding adversarial subspaces, which are (i) oblivious attacks and LID analysis using adversarial examples with different confidence levels; and (ii) black-box transfer attacks. For (i), we find that the performance of LID is very sensitive to the confidence parameter deployed by an attack, and the LID learned from ensembles of adversarial examples with varying confidence levels surprisingly gives poor performance. For (ii), we find that when adversarial examples are crafted from another DNN model, LID is ineffective in characterizing their adversarial subspaces. These two findings together suggest the limited capability of LID in characterizing the subspaces of adversarial examples.

arXiv commentAccepted to ICLR 2018 Worshop

References in corpus (7)

• Explaining and Harnessing Adversarial Examples
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
• Attacking the Madry Defense Model with $L_1$-based Adversarial Examples
• EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples

Cited by in corpus (10)

• On Evaluating Adversarial Robustness
• Adversarial Examples in Deep Learning: Characterization and Divergence
• ML-LOO: Detecting Adversarial Examples with Feature Attribution
• On the Limitation of MagNet Defense against $L_1$-based Adversarial Examples
• Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models
• LiBRe: A Practical Bayesian Approach to Adversarial Detection
• $n$-ML: Mitigating Adversarial Examples via Ensembles of Topologically Manipulated Classifiers
• On The Utility of Conditional Generation Based Mutual Information for Characterizing Adversarial Subspaces
• Linking average- and worst-case perturbation robustness via class selectivity and dimensionality
• Weighted Average Precision: Adversarial Example Detection in the Visual Perception of Autonomous Vehicles