Density of States Estimation for Out-of-Distribution Detection

Abstract

Perhaps surprisingly, recent studies have shown probabilistic model likelihoods have poor specificity for out-of-distribution (OOD) detection and often assign higher likelihoods to OOD data than in-distribution data. To ameliorate this issue we propose DoSE, the density of states estimator. Drawing on the statistical physics notion of ``density of states,'' the DoSE decision rule avoids direct comparison of model probabilities, and instead utilizes the ``probability of the model probability,'' or indeed the frequency of any reasonable statistic. The frequency is calculated using nonparametric density estimators (e.g., KDE and one-class SVM) which measure the typicality of various model statistics given the training data and from which we can flag test points with low typicality as anomalous. Unlike many other methods, DoSE requires neither labeled data nor OOD examples. DoSE is modular and can be trivially applied to any existing, trained model. We demonstrate DoSE's state-of-the-art performance against other unsupervised OOD detectors on previously established ``hard'' benchmarks.

arXiv commentSubmitted to NeurIPS. Corrected footnote from: "34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada" to "Preprint. Under review."

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Scikit-learn: Machine Learning in Python
• Intriguing properties of neural networks
• Deep Anomaly Detection with Outlier Exposure
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Input complexity and out-of-distribution detection with likelihood-based generative models
• Reducing Network Agnostophobia
• Uncertainty in the Variational Information Bottleneck
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality

Cited by in corpus (7)

• Outlier Exposure with Confidence Control for Out-of-Distribution Detection
• Learning and Evaluating Representations for Deep One-class Classification
• Perfect density models cannot guarantee anomaly detection
• The Hidden Uncertainty in a Neural Networks Activations
• Bayesian OOD detection with aleatoric uncertainty and outlier exposure
• Task-agnostic Continual Learning with Hybrid Probabilistic Models
• On Out-of-distribution Detection with Energy-based Models