Implicit Generative Modeling of Random Noise during Training for Adversarial Robustness

Abstract

We introduce a Noise-based prior Learning (NoL) approach for training neural networks that are intrinsically robust to adversarial attacks. We find that the implicit generative modeling of random noise with the same loss function used during posterior maximization, improves a model's understanding of the data manifold furthering adversarial robustness. We evaluate our approach's efficacy and provide a simplistic visualization tool for understanding adversarial data, using Principal Component Analysis. Our analysis reveals that adversarial robustness, in general, manifests in models with higher variance along the high-ranked principal components. We show that models learnt with our approach perform remarkably well against a wide-range of attacks. Furthermore, combining NoL with state-of-the-art adversarial training extends the robustness of a model, even beyond what it is adversarially trained for, in both white-box and black-box attack scenarios.

arXiv commentPreliminary version of this work accepted at ICML 2019 (Workshop on Uncertainty and Robustness in Deep Learning)

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Ensemble Adversarial Training: Attacks and Defenses
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• The Space of Transferable Adversarial Examples
• Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN
• Adversarial Perturbations Against Deep Neural Networks for Malware Classification
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Adversarial Transformation Networks: Learning to Generate Adversarial Examples
• Foveation-based Mechanisms Alleviate Adversarial Examples
• Early Methods for Detecting Adversarial Images
• Robustness of classifiers: from adversarial to random noise
• Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN
• Attacking the Madry Defense Model with $L_1$-based Adversarial Examples

Cited by in corpus (1)

• Discretization based Solutions for Secure Machine Learning against Adversarial Attacks