Trust but Verify: An Information-Theoretic Explanation for the Adversarial Fragility of Machine Learning Systems, and a General Defense against Adversarial Attacks
arXiv:1905.11381
Abstract
Deep-learning based classification algorithms have been shown to be susceptible to adversarial attacks: minor changes to the input of classifiers can dramatically change their outputs, while being imperceptible to humans. In this paper, we present a simple hypothesis about a feature compression property of artificial intelligence (AI) classifiers and present theoretical arguments to show that this hypothesis successfully accounts for the observed fragility of AI classifiers to small adversarial perturbations. Drawing on ideas from information and coding theory, we propose a general class of defenses for detecting classifier errors caused by abnormally small input perturbations. We further show theoretical guarantees for the performance of this detection method. We present experimental results with (a) a voice recognition system, and (b) a digit recognition system using the MNIST database, to demonstrate the effectiveness of the proposed defense methods. The ideas in this paper are motivated by a simple analogy between AI classifiers and the standard Shannon model of a communication system.
arXiv comment44 Pages, 2 Theorems, 35 Figures, 29 Tables. arXiv admin note: substantial text overlap with arXiv:1901.09413
References in corpus (32)
- Explaining and Harnessing Adversarial Examples
- Ensemble Adversarial Training: Attacks and Defenses
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- Opening the Black Box of Deep Neural Networks via Information
- Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
- Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
- Defensive Distillation is Not Robust to Adversarial Examples
- Adversarial Examples: Attacks and Defenses for Deep Learning
- Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
- A Boundary Tilting Persepective on the Phenomenon of Adversarial Examples
- The Robust Manifold Defense: Adversarial Training using Generative Models
- Rademacher Complexity for Adversarially Robust Generalization
- Guessing Smart: Biased Sampling for Efficient Black-Box Adversarial Attacks
- Adversarial Examples for Semantic Segmentation and Object Detection
- Feature Denoising for Improving Adversarial Robustness
- Adversarial Manipulation of Deep Representations
- Discretization based Solutions for Secure Machine Learning against Adversarial Attacks
- A Survey of Safety and Trustworthiness of Deep Neural Networks: Verification, Testing, Adversarial Attack and Defence, and Interpretability
- DARCCC: Detecting Adversaries by Reconstruction from Class Conditional Capsules
- Robust Neural Networks using Randomized Adversarial Training
- Deep-RBF Networks Revisited: Robust Classification with Rejection
- Discrete Adversarial Attacks and Submodular Optimization with Applications to Text Classification
- Enhancing the Robustness of Deep Neural Networks by Boundary Conditional GAN
- Seeing isn't Believing: Practical Adversarial Attack Against Object Detectors
- Simple Physical Adversarial Examples against End-to-End Autonomous Driving Models
- Efficient Decision-based Black-box Adversarial Attacks on Face Recognition
- Mathematical Analysis of Adversarial Attacks
- Improving Adversarial Robustness via Guided Complement Entropy
- GanDef: A GAN based Adversarial Training Defense for Neural Network Classifier
- Weight Map Layer for Noise and Adversarial Attack Robustness
- ZK-GanDef: A GAN based Zero Knowledge Adversarial Training Defense for Neural Networks
- Disentangled Deep Autoencoding Regularization for Robust Image Classification