Adversarial Neural Pruning with Latent Vulnerability Suppression
arXiv:1908.04355
Abstract
Despite the remarkable performance of deep neural networks on various computer vision tasks, they are known to be susceptible to adversarial perturbations, which makes it challenging to deploy them in real-world safety-critical applications. In this paper, we conjecture that the leading cause of adversarial vulnerability is the distortion in the latent feature space, and provide methods to suppress them effectively. Explicitly, we define \emph{vulnerability} for each latent feature and then propose a new loss for adversarial learning, \emph{Vulnerability Suppression (VS)} loss, that aims to minimize the feature-level vulnerability during training. We further propose a Bayesian framework to prune features with high vulnerability to reduce both vulnerability and loss on adversarial samples. We validate our \emph{Adversarial Neural Pruning with Vulnerability Suppression (ANP-VS)} method on multiple benchmark datasets, on which it not only obtains state-of-the-art adversarial robustness but also improves the performance on clean examples, using only a fraction of the parameters used by the full network. Further qualitative analysis suggests that the improvements come from the suppression of feature-level vulnerability.
arXiv commentAccepted to ICML 2020. Code available at https://github.com/divyam3897/ANP_VS
References in corpus (19)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- End to End Learning for Self-Driving Cars
- The information bottleneck method
- Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- Theoretically Principled Trade-off between Robustness and Accuracy
- Adversarially Robust Generalization Requires More Data
- Variational Dropout and the Local Reparameterization Trick
- Using Pre-Training Can Improve Model Robustness and Uncertainty
- Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
- Variational Dropout Sparsifies Deep Neural Networks
- Structured Bayesian Pruning via Log-Normal Multiplicative Noise
- Memory Bounded Deep Convolutional Networks
- Compressing Neural Networks using the Variational Information Bottleneck
- Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network
- Sparse DNNs with Improved Adversarial Robustness
- Evaluating and Understanding the Robustness of Adversarial Logit Pairing
- Stick-Breaking Variational Autoencoders
- A Spectral View of Adversarially Robust Features