Further Analysis of Outlier Detection with Deep Generative Models
arXiv:2010.13064
Abstract
The recent, counter-intuitive discovery that deep generative models (DGMs) can frequently assign a higher likelihood to outliers has implications for both outlier detection applications as well as our overall understanding of generative modeling. In this work, we present a possible explanation for this phenomenon, starting from the observation that a model's typical set and high-density region may not conincide. From this vantage point we propose a novel outlier test, the empirical success of which suggests that the failure of existing likelihood-based outlier tests does not necessarily imply that the corresponding generative model is uncalibrated. We also conduct additional experiments to help disentangle the impact of low-level texture versus high-level semantics in differentiating outliers. In aggregate, these results suggest that modifications to the standard evaluation practices and benchmarks commonly applied in the literature are needed.
arXiv commentNeurIPS 2020
References in corpus (16)
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Confounding variables can degrade generalization performance of radiological deep learning models
- NICE: Non-linear Independent Components Estimation
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
- Deep Anomaly Detection with Outlier Exposure
- Deep Anomaly Detection Using Geometric Transformations
- WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
- BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling
- Classification-Based Anomaly Detection for General Data
- Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
- Concentration of the information in data with log-concave distributions
- Uncertainty in the Variational Information Bottleneck
- Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
- Hybrid Models with Deep and Invertible Features
- The Functional Neural Process