Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
arXiv:1906.02994
Abstract
Recent work has shown that deep generative models can assign higher likelihood to out-of-distribution data sets than to their training data (Nalisnick et al., 2019; Choi et al., 2019). We posit that this phenomenon is caused by a mismatch between the model's typical set and its areas of high probability density. In-distribution inputs should reside in the former but not necessarily in the latter, as previous work has presumed. To determine whether or not inputs reside in the typical set, we propose a statistically principled, easy-to-implement test using the empirical distribution of model likelihoods. The test is model agnostic and widely applicable, only requiring that the likelihood can be computed or closely approximated. We report experiments showing that our procedure can successfully detect the out-of-distribution sets in several of the challenging cases reported by Nalisnick et al. (2019).
References in corpus (9)
- Auto-Encoding Variational Bayes
- A Conceptual Introduction to Hamiltonian Monte Carlo
- Deep Anomaly Detection with Outlier Exposure
- Learning in Implicit Generative Models
- WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
- A Kernel Test of Goodness of Fit
- Distribution Matching in Variational Inference
- Are generative deep models for novelty detection truly better?
- A Less Biased Evaluation of Out-of-distribution Sample Detectors
Cited by in corpus (38)
- CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
- Learning and Evaluating Representations for Deep One-class Classification
- Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
- Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder
- A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
- Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts
- Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
- Anomaly localization by modeling perceptual features
- Feature Space Singularity for Out-of-Distribution Detection
- Perfect density models cannot guarantee anomaly detection
- Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection
- Unsupervised Out-of-Distribution Detection with Batch Normalization
- Density of States Estimation for Out-of-Distribution Detection
- Novelty Detection Via Blurring
- The Hidden Uncertainty in a Neural Networks Activations
- Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders
- Further Analysis of Outlier Detection with Deep Generative Models
- Hybrid Models for Open Set Recognition
- Hierarchical VAEs Know What They Don't Know
- Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection
- Detecting Out-of-distribution Samples via Variational Auto-encoder with Reliable Uncertainty Estimation
- Do We Really Need to Learn Representations from In-domain Data for Outlier Detection?
- Detecting Anomalous Event Sequences with Temporal Point Processes
- On the Out-of-distribution Generalization of Probabilistic Image Modelling
- Task-agnostic Continual Learning with Hybrid Probabilistic Models
- Bigeminal Priors Variational auto-encoder
- Efficient Learning of Generative Models via Finite-Difference Score Matching
- Likelihood Assignment for Out-of-Distribution Inputs in Deep Generative Models is Sensitive to Prior Distribution Choice
- Are Bayesian neural networks intrinsically good at out-of-distribution detection?
- On Out-of-distribution Detection with Energy-based Models
- OSOA: One-Shot Online Adaptation of Deep Generative Models for Lossless Compression
- Learning Bijective Feature Maps for Linear ICA
- Robust Out-of-Distribution Detection on Deep Probabilistic Generative Models
- Towards Robust Active Feature Acquisition
- Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More
- Entropic Issues in Likelihood-Based OOD Detection
- Context selectivity with dynamic availability enables lifelong continual learning
- Kullback-Leibler Divergence-Based Out-of-Distribution Detection with Flow-Based Generative Models