Do Deep Generative Models Know What They Don't Know?

Abstract

A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data. A plethora of work has demonstrated that it is easy to find or synthesize inputs for which a neural network is highly confident yet wrong. Generative models are widely viewed to be robust to such mistaken confidence as modeling the density of the input features can be used to detect novel, out-of-distribution inputs. In this paper we challenge this assumption. We find that the density learned by flow-based models, VAEs, and PixelCNNs cannot distinguish images of common objects such as dogs, trucks, and horses (i.e. CIFAR-10) from those of house numbers (i.e. SVHN), assigning a higher likelihood to the latter when the model is trained on the former. Moreover, we find evidence of this phenomenon when pairing several popular image data sets: FashionMNIST vs MNIST, CelebA vs SVHN, ImageNet vs CIFAR-10 / CIFAR-100 / SVHN. To investigate this curious behavior, we focus analysis on flow-based generative models in particular since they are trained and evaluated via the exact marginal likelihood. We find such behavior persists even when we restrict the flows to constant-volume transformations. These transformations admit some theoretical analysis, and we show that the difference in likelihoods can be explained by the location and variances of the data and the model curvature. Our results caution against using the density estimates from deep generative models to identify inputs similar to the training distribution until their behavior for out-of-distribution inputs is better understood.

arXiv commentICLR 2019

References in corpus (10)

• Auto-Encoding Variational Bayes
• WaveNet: A Generative Model for Raw Audio
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• NICE: Non-linear Independent Components Estimation
• Deep Anomaly Detection with Outlier Exposure
• Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
• Amortised MAP Inference for Image Super-resolution
• Adversarial Images for Variational Autoencoders
• Distribution Matching in Variational Inference
• Are generative deep models for novelty detection truly better?

Cited by in corpus (24)

• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
• Learning Open Set Network with Discriminative Reciprocal Points
• The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
• The Conditional Entropy Bottleneck
• Pseudo-healthy synthesis with pathology disentanglement and adversarial learning
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
• Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
• Detecting Out-of-Distribution Inputs in Deep Neural Networks Using an Early-Layer Output
• Monitoring and explainability of models in production
• A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities
• Feature Space Singularity for Out-of-Distribution Detection
• Learning image representations for anomaly detection: application to discovery of histological alterations in drug development
• Perfect density models cannot guarantee anomaly detection
• Learning Densities in Feature Space for Reliable Segmentation of Indoor Scenes
• Visual Anomaly Detection for Images: A Survey
• A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks
• Hybrid Models for Open Set Recognition
• Stationary Activations for Uncertainty Calibration in Deep Learning
• Detecting Out-of-distribution Samples via Variational Auto-encoder with Reliable Uncertainty Estimation
• Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection
• Rectangular Flows for Manifold Learning
• Data-Efficient Methods for Dialogue Systems
• Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More