Trust but Verify: An Information-Theoretic Explanation for the Adversarial Fragility of Machine Learning Systems, and a General Defense against Adversarial Attacks
arXiv:1905.11381
Abstract
Deep-learning based classification algorithms have been shown to be susceptible to adversarial attacks: minor changes to the input of classifiers can dramatically change their outputs, while being imperceptible to humans. In this paper, we present a simple hypothesis about a feature compression property of artificial intelligence (AI) classifiers and present theoretical arguments to show that this hypothesis successfully accounts for the observed fragility of AI classifiers to small adversarial perturbations. Drawing on ideas from information and coding theory, we propose a general class of defenses for detecting classifier errors caused by abnormally small input perturbations. We further show theoretical guarantees for the performance of this detection method. We present experimental results with (a) a voice recognition system, and (b) a digit recognition system using the MNIST database, to demonstrate the effectiveness of the proposed defense methods. The ideas in this paper are motivated by a simple analogy between AI classifiers and the standard Shannon model of a communication system.
44 Pages, 2 Theorems, 35 Figures, 29 Tables. arXiv admin note: substantial text overlap with arXiv:1901.09413
References in corpus (10)
- Explaining and Harnessing Adversarial Examples
- Defensive Distillation is Not Robust to Adversarial Examples
- A Boundary Tilting Persepective on the Phenomenon of Adversarial Examples
- Discretization based Solutions for Secure Machine Learning against Adversarial Attacks
- Enhancing the Robustness of Deep Neural Networks by Boundary Conditional GAN
- Simple Physical Adversarial Examples against End-to-End Autonomous Driving Models
- Efficient Decision-based Black-box Adversarial Attacks on Face Recognition
- GanDef: A GAN based Adversarial Training Defense for Neural Network Classifier
- ZK-GanDef: A GAN based Zero Knowledge Adversarial Training Defense for Neural Networks
- Disentangled Deep Autoencoding Regularization for Robust Image Classification