Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses

Abstract

Convolutional Neural Networks have been shown to be vulnerable to adversarial examples, which are known to locate in subspaces close to where normal data lies but are not naturally occurring and of low probability. In this work, we investigate the potential effect defense techniques have on the geometry of the likelihood landscape - likelihood of the input images under the trained model. We first propose a way to visualize the likelihood landscape leveraging an energy-based model interpretation of discriminative classifiers. Then we introduce a measure to quantify the flatness of the likelihood landscape. We observe that a subset of adversarial defense techniques results in a similar effect of flattening the likelihood landscape. We further explore directly regularizing towards a flat landscape for adversarial robustness.

arXiv commentECCV 2020 Workshop on Adversarial Robustness in the Real World

References in corpus (21)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Theoretically Principled Trade-off between Robustness and Accuracy
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
• Countering Adversarial Images using Input Transformations
• Detecting Adversarial Samples from Artifacts
• Robustness May Be at Odds with Accuracy
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• Robust Large Margin Deep Neural Networks
• Qualitatively characterizing neural network optimization problems
• Mitigating Adversarial Effects Through Randomization
• Adversarial Robustness through Local Linearization
• Compression Artifacts Removal Using Convolutional Neural Networks
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
• Adversarial Robustness as a Prior for Learned Representations
• Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
• Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks
• Robust Learning with Jacobian Regularization