Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder

Abstract

Deep probabilistic generative models enable modeling the likelihoods of very high dimensional data. An important application of generative modeling should be the ability to detect out-of-distribution (OOD) samples by setting a threshold on the likelihood. However, some recent studies show that probabilistic generative models can, in some cases, assign higher likelihoods on certain types of OOD samples, making the OOD detection rules based on likelihood threshold problematic. To address this issue, several OOD detection methods have been proposed for deep generative models. In this paper, we make the observation that many of these methods fail when applied to generative models based on Variational Auto-encoders (VAE). As an alternative, we propose Likelihood Regret, an efficient OOD score for VAEs. We benchmark our proposed method over existing approaches, and empirical results suggest that our method obtains the best overall OOD detection performances when applied to VAEs.

arXiv commentNeurIPS 2020

References in corpus (21)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• WaveNet: A Generative Model for Raw Audio
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Conditional Image Generation with PixelCNN Decoders
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Variational Graph Auto-Encoders
• Deep Anomaly Detection with Outlier Exposure
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Input complexity and out-of-distribution detection with likelihood-based generative models
• BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling
• Diagnosing and Enhancing VAE Models
• Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
• Improving Reconstruction Autoencoder Out-of-distribution Detection with Mahalanobis Distance
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
• Generative Latent Flow
• Unsupervised Out-of-Distribution Detection with Batch Normalization
• Anomalous Example Detection in Deep Learning: A Survey

Cited by in corpus (13)

• A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges
• Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data
• Hierarchical VAEs Know What They Don't Know
• Detecting Out-of-distribution Samples via Variational Auto-encoder with Reliable Uncertainty Estimation
• InFlow: Robust outlier detection utilizing Normalizing Flows
• Sliced Iterative Normalizing Flows
• CVAD: A generic medical anomaly detector based on Cascade VAE
• Model-agnostic out-of-distribution detection using combined statistical tests
• Unsupervised Anomaly Detection From Semantic Similarity Scores
• Autoencoding Under Normalization Constraints
• Beyond Confidence: Adaptive Abstention in Dual-Threshold Conformal Prediction for Autonomous System Perception
• Entropic Issues in Likelihood-Based OOD Detection
• GalilAI: Out-of-Task Distribution Detection using Causal Active Experimentation for Safe Transfer RL