Understanding the Limitations of Conditional Generative Models

Abstract

Class-conditional generative models hold promise to overcome the shortcomings of their discriminative counterparts. They are a natural choice to solve discriminative tasks in a robust manner as they jointly optimize for predictive performance and accurate modeling of the input distribution. In this work, we investigate robust classification with likelihood-based generative models from a theoretical and practical perspective to investigate if they can deliver on their promises. Our analysis focuses on a spectrum of robustness properties: (1) Detection of worst-case outliers in the form of adversarial examples; (2) Detection of average-case outliers in the form of ambiguous inputs and (3) Detection of incorrectly labeled in-distribution inputs. Our theoretical result reveals that it is impossible to guarantee detectability of adversarially-perturbed inputs even for near-optimal generative classifiers. Experimentally, we find that while we are able to train robust models for MNIST, robustness completely breaks down on CIFAR10. We relate this failure to various undesirable model properties that can be traced to the maximum likelihood training objective. Despite being a common choice in the literature, our results indicate that likelihood-based conditional generative models may are surprisingly ineffective for robust classification.

References in corpus (8)

• Conditional Image Generation with PixelCNN Decoders
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• Towards the first adversarially robust neural network model on MNIST
• Why do deep convolutional networks generalize so poorly to small image transformations?
• Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models
• DARCCC: Detecting Adversaries by Reconstruction from Class Conditional Capsules
• Hybrid Models with Deep and Invertible Features

Cited by in corpus (8)

• Shortcut Learning in Deep Neural Networks
• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Perfect density models cannot guarantee anomaly detection
• Understanding and Mitigating Exploding Inverses in Invertible Neural Networks
• Evaluating State-of-the-Art Classification Models Against Bayes Optimality
• Score-Based Generative Classifiers
• Conditional Generative Modeling via Learning the Latent Space
• Learning to Separate Clusters of Adversarial Representations for Robust Adversarial Detection