One Bit Matters: Understanding Adversarial Examples as the Abuse of Redundancy
arXiv:1810.09650
Abstract
Despite the great success achieved in machine learning (ML), adversarial examples have caused concerns with regards to its trustworthiness: A small perturbation of an input results in an arbitrary failure of an otherwise seemingly well-trained ML model. While studies are being conducted to discover the intrinsic properties of adversarial examples, such as their transferability and universality, there is insufficient theoretic analysis to help understand the phenomenon in a way that can influence the design process of ML experiments. In this paper, we deduce an information-theoretic model which explains adversarial attacks as the abuse of feature redundancies in ML algorithms. We prove that feature redundancy is a necessary condition for the existence of adversarial examples. Our model helps to explain some major questions raised in many anecdotal studies on adversarial examples. Our theory is backed up by empirical measurements of the information content of benign and adversarial examples on both image and text datasets. Our measurements show that typical adversarial examples introduce just enough redundancy to overflow the decision making of an ML model trained on corresponding benign examples. We conclude with actionable recommendations to improve the robustness of machine learners against adversarial examples.
References in corpus (19)
- Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- Delving into Transferable Adversarial Examples and Black-box Attacks
- Robust Physical-World Attacks on Deep Learning Models
- Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
- The Space of Transferable Adversarial Examples
- Detecting Adversarial Samples from Artifacts
- Distributional Smoothing with Virtual Adversarial Training
- Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
- Delving into adversarial attacks on deep policies
- DeepFool: a simple and accurate method to fool deep neural networks
- Fast Feature Fool: A data independent approach to universal adversarial perturbations
- Universal adversarial perturbations
- HotFlip: White-Box Adversarial Examples for Text Classification
- A Theoretical Framework for Robustness of (Deep) Classifiers against Adversarial Examples
- Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples
- A Capacity Scaling Law for Artificial Neural Networks
- Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification
- The Helmholtz Method: Using Perceptual Compression to Reduce Machine Learning Complexity