Perfect density models cannot guarantee anomaly detection

Abstract

Thanks to the tractability of their likelihood, several deep generative models show promise for seemingly straightforward but important applications like anomaly detection, uncertainty estimation, and active learning. However, the likelihood values empirically attributed to anomalies conflict with the expectations these proposed applications suggest. In this paper, we take a closer look at the behavior of distribution densities through the lens of reparametrization and show that these quantities carry less meaningful information than previously thought, beyond estimation issues or the curse of dimensionality. We conclude that the use of these likelihoods for anomaly detection relies on strong and implicit hypotheses, and highlight the necessity of explicitly formulating these assumptions for reliable anomaly detection.

arXiv commentAccepted to the Special Issue "Probabilistic Methods for Deep Learning" of the Journal Entropy. 14 pages and 10 figures in main content, 4 pages of bibliography, and 2 pages in Appendix

References in corpus (54)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Array Programming with NumPy
• Auto-Encoding Variational Bayes
• The NumPy array: a structure for efficient numerical computation
• Intriguing properties of neural networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• Normalizing Flows: An Introduction and Review of Current Methods
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• A Unifying Review of Deep and Shallow Anomaly Detection
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Density estimation using Real NVP
• VIME: Variational Information Maximizing Exploration
• Energy-based Out-of-distribution Detection
• A note on the evaluation of generative models
• Deep Anomaly Detection with Outlier Exposure
• NVAE: A Deep Hierarchical Variational Autoencoder
• Normalizing Flows for Probabilistic Modeling and Inference
• Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• Towards a Critical Race Methodology in Algorithmic Fairness
• Image Transformer
• Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Input complexity and out-of-distribution detection with likelihood-based generative models
• Crypto-Nets: Neural Networks over Encrypted Data
• Does Object Recognition Work for Everyone?
• A Deep and Tractable Density Estimator
• Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
• EX2: Exploration with Exemplar Models for Deep Reinforcement Learning
• Do Deep Generative Models Know What They Don't Know?
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
• Contrastive Training for Improved Out-of-Distribution Detection
• Curiosity-Driven Experience Prioritization via Density Estimation
• Against Scale: Provocations and Resistances to Scale Thinking
• Why Normalizing Flows Fail to Detect Out-of-Distribution Data
• Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Features
• Hybrid Discriminative-Generative Training via Contrastive Learning
• Understanding and Mitigating Exploding Inverses in Invertible Neural Networks
• Understanding Failures in Out-of-Distribution Detection with Deep Generative Models
• AI and the Everything in the Whole Wide World Benchmark
• Density of States Estimation for Out-of-Distribution Detection
• Understanding the Limitations of Conditional Generative Models
• Further Analysis of Outlier Detection with Deep Generative Models
• Understanding the (un)interpretability of natural image distributions using generative models
• Hybrid Models for Open Set Recognition
• Deep Generative Models Strike Back! Improving Understanding and Evaluation in Light of Unmet Expectations for OoD Data
• Reparametrization Invariance in non-parametric Causal Discovery

Cited by in corpus (10)

• Anomaly Detection under Coordinate Transformations
• Deep machine learning for meteor monitoring: advances with transfer learning and gradient-weighted class activation mapping
• Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification
• Innovations Autoencoder and its Application in One-class Anomalous Sequence Detection
• Anomaly detection with flow-based fast calorimeter simulators
• Object classification on video data of meteors and meteor-like phenomena: algorithm and data
• Do We Really Need to Learn Representations from In-domain Data for Outlier Detection?
• New Methods and Datasets for Group Anomaly Detection From Fundamental Physics
• Autoencoding Under Normalization Constraints
• Entropic Issues in Likelihood-Based OOD Detection