Deep Anomaly Detection with Outlier Exposure

Abstract

It is important to detect anomalous inputs when deploying machine learning systems. The use of larger and more complex inputs in deep learning magnifies the difficulty of distinguishing between anomalous and in-distribution examples. At the same time, diverse image and text data are available in enormous quantities. We propose leveraging these data to improve deep anomaly detection by training anomaly detectors against an auxiliary dataset of outliers, an approach we call Outlier Exposure (OE). This enables anomaly detectors to generalize and detect unseen anomalies. In extensive experiments on natural language processing and small- and large-scale vision tasks, we find that Outlier Exposure significantly improves detection performance. We also observe that cutting-edge generative models trained on CIFAR-10 may assign higher likelihoods to SVHN images than to CIFAR-10 images; we use OE to mitigate this issue. We also analyze the flexibility and robustness of Outlier Exposure, and identify characteristics of the auxiliary dataset that improve performance.

ICLR 2019; PyTorch code available at https://github.com/hendrycks/outlier-exposure

Cited by in corpus (189)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Energy-based Out-of-distribution Detection
• Adversarial Reciprocal Points Learning for Open Set Recognition
• Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
• Sub-Image Anomaly Detection with Deep Pyramid Correspondences
• Measuring Massive Multitask Language Understanding
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Learning Open Set Network with Discriminative Reciprocal Points
• Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
• Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
• Scaling Out-of-Distribution Detection for Real-World Settings
• Measuring Calibration in Deep Learning
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Input complexity and out-of-distribution detection with likelihood-based generative models
• Deep Nearest Neighbor Anomaly Detection
• Exploring the Limits of Out-of-Distribution Detection
• Explainable Deep One-Class Classification
• Outlier Exposure with Confidence Control for Out-of-Distribution Detection
• Classification-Based Anomaly Detection for General Data
• Implicit Generation and Generalization in Energy-Based Models
• Learning and Evaluating Representations for Deep One-class Classification
• Unsolved Problems in ML Safety
• Verified Uncertainty Calibration
• Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder
• SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation
• Contrastive Out-of-Distribution Detection for Pretrained Transformers
• Do Deep Generative Models Know What They Don't Know?
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
• Contrastive Training for Improved Out-of-Distribution Detection
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Out-of-distribution Detection in Classifiers via Generation
• SSD: A Unified Framework for Self-Supervised Outlier Detection
• Attribute Restoration Framework for Anomaly Detection
• A Survey on Open Set Recognition
• Why is the Mahalanobis Distance Effective for Anomaly Detection?
• Open-Set Recognition: a Good Closed-Set Classifier is All You Need?
• Identifying Causal-Effect Inference Failure with Uncertainty-Aware Models
• Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data
• Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
• Description and Discussion on DCASE 2021 Challenge Task 2: Unsupervised Anomalous Sound Detection for Machine Condition Monitoring under Domain Shifted Conditions
• Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks
• Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Features
• Robust Out-of-distribution Detection for Neural Networks
• Open-set Label Noise Can Improve Robustness Against Inherent Label Noise
• Entropic Out-of-Distribution Detection: Seamless Detection of Unknown Examples
• Deep Semi-Supervised Anomaly Detection
• Pretrained Transformers Improve Out-of-Distribution Robustness
• Feature Space Singularity for Out-of-Distribution Detection
• Uncertainty-Aware Reliable Text Classification
• Improving colonoscopy lesion classification using semi-supervised deep learning
• Perfect density models cannot guarantee anomaly detection
• OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers
• Deep CNN-based Multi-task Learning for Open-Set Recognition
• Unsupervised in-distribution anomaly detection of new physics through conditional density estimation
• Neural Transformation Learning for Deep Anomaly Detection Beyond Images
• Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices
• A Less Biased Evaluation of Out-of-distribution Sample Detectors
• A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges
• Deep Learning for Insider Threat Detection: Review, Challenges and Opportunities
• On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation
• Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection
• Unsupervised Out-of-Distribution Detection with Batch Normalization
• AutoOD: Automated Outlier Detection via Curiosity-guided Search and Self-imitation Learning
• Normality-Calibrated Autoencoder for Unsupervised Anomaly Detection on Data Contamination
• Density of States Estimation for Out-of-Distribution Detection
• Tail of Distribution GAN (TailGAN): Generative-Adversarial-Network-Based Boundary Formation
• Novelty Detection Via Blurring
• Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity
• Rethinking Assumptions in Deep Anomaly Detection
• Attacking Adversarial Attacks as A Defense
• Boundary of Distribution Support Generator (BDSG): Sample Generation on the Boundary
• SoK: Machine Learning Governance
• Are all outliers alike? On Understanding the Diversity of Outliers for Detecting OODs
• Real-time Out-of-distribution Detection in Learning-Enabled Cyber-Physical Systems
• Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation
• Further Analysis of Outlier Detection with Deep Generative Models
• ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining
• Anomalous Example Detection in Deep Learning: A Survey
• Accelerating Monte Carlo Bayesian Inference via Approximating Predictive Uncertainty over Simplex
• Analysis of Confident-Classifiers for Out-of-distribution Detection
• Distribution-free binary classification: prediction sets, confidence intervals and calibration
• MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space
• Hierarchical VAEs Know What They Don't Know
• Detecting OODs as datapoints with High Uncertainty
• Out-of-domain Detection for Natural Language Understanding in Dialog Systems
• No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets
• A Critical Evaluation of Open-World Machine Learning
• Negative Data Augmentation
• Likelihood Ratios for Out-of-Distribution Detection
• Hybrid Models for Open Set Recognition
• Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
• Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers
• Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection
• DropCluster: A structured dropout for convolutional networks
• DROCC: Deep Robust One-Class Classification
• Multiscale Score Matching for Out-of-Distribution Detection
• Efficacy of Pixel-Level OOD Detection for Semantic Segmentation
• SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detection
• Anomalous Sound Detection as a Simple Binary Classification Problem with Careful Selection of Proxy Outlier Examples
• Detecting Out-of-distribution Samples via Variational Auto-encoder with Reliable Uncertainty Estimation
• $k$Folden: $k$-Fold Ensemble for Out-Of-Distribution Detection
• Revisiting Classical Bagging with Modern Transfer Learning for On-the-fly Disaster Damage Detector
• Mobile Traffic Classification through Physical Channel Fingerprinting: a Deep Learning Approach
• Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks
• Selective Question Answering under Domain Shift
• Flow-based Self-supervised Density Estimation for Anomalous Sound Detection
• Bounds all around: training energy-based models with bidirectional bounds
• Hardness Sampling for Self-Training Based Transductive Zero-Shot Learning
• Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urban Driving Scenes
• InFlow: Robust outlier detection utilizing Normalizing Flows
• Regularizing Attention Networks for Anomaly Detection in Visual Question Answering
• A Simple Framework for Uncertainty in Contrastive Learning
• A t-distribution based operator for enhancing out of distribution robustness of neural network classifiers
• Bayesian OOD detection with aleatoric uncertainty and outlier exposure
• Provably Robust Detection of Out-of-distribution Data (almost) for free
• Learnable Uncertainty under Laplace Approximations
• Self-supervised Robust Object Detectors from Partially Labelled Datasets
• Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data
• Thinkback: Task-SpecificOut-of-Distribution Detection
• Deep Random Projection Outlyingness for Unsupervised Anomaly Detection
• An Algorithm for Out-Of-Distribution Attack to Neural Network Encoder
• SCOUT: Self-aware Discriminant Counterfactual Explanations
• On the Out-of-distribution Generalization of Probabilistic Image Modelling
• Toward Metrics for Differentiating Out-of-Distribution Sets
• ProtoInfoMax: Prototypical Networks with Mutual Information Maximization for Out-of-Domain Detection
• Enhanced Isotropy Maximization Loss: Seamless and High-Performance Out-of-Distribution Detection Simply Replacing the SoftMax Loss
• Bigeminal Priors Variational auto-encoder
• Bridging In- and Out-of-distribution Samples for Their Better Discriminability
• Unsupervised Out-of-Domain Detection via Pre-trained Transformers
• Robustness via Cross-Domain Ensembles
• Masked Contrastive Learning for Anomaly Detection
• Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learning
• Likelihood Assignment for Out-of-Distribution Inputs in Deep Generative Models is Sensitive to Prior Distribution Choice
• Enhancing Unsupervised Anomaly Detection with Score-Guided Network
• Topological Uncertainty: Monitoring trained neural networks through persistence of activation graphs
• Adaptive Deep Learning for Entity Resolution by Risk Analysis
• Being a Bit Frequentist Improves Bayesian Neural Networks
• Deep Anomaly Detection by Residual Adaptation
• An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
• Shifting Transformation Learning for Out-of-Distribution Detection
• Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog
• Exploring Vicinal Risk Minimization for Lightweight Out-of-Distribution Detection
• Out-Of-Distribution Detection With Subspace Techniques And Probabilistic Modeling Of Features
• Energy-Based Anomaly Detection and Localization
• Unsupervised Class-Incremental Learning Through Confusion
• A statistical framework for efficient out of distribution detection in deep neural networks
• Removing Undesirable Feature Contributions Using Out-of-Distribution Data
• Toward Scalable and Unified Example-based Explanation and Outlier Detection
• OMASGAN: Out-of-Distribution Minimum Anomaly Score GAN for Sample Generation on the Boundary
• Robust Out-of-Distribution Detection on Deep Probabilistic Generative Models
• Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training
• Towards Robust Active Feature Acquisition
• DOI: Divergence-based Out-of-Distribution Indicators via Deep Generative Models
• Unsupervised Performance Analysis of 3D Face Alignment with a Statistically Robust Confidence Test
• Discriminative Multi-level Reconstruction under Compact Latent Space for One-Class Novelty Detection
• A Survey on Proactive Customer Care: Enabling Science and Steps to Realize it
• GCCN: Global Context Convolutional Network
• Signature-Graph Networks
• Being Single Has Benefits. Instance Poisoning to Deceive Malware Classifiers
• Out-distribution aware Self-training in an Open World Setting
• Autoencoding Under Normalization Constraints
• PnPOOD : Out-Of-Distribution Detection for Text Classification via Plug andPlay Data Augmentation
• Out of distribution detection for skin and malaria images
• Kullback-Leibler Divergence-Based Out-of-Distribution Detection with Flow-Based Generative Models
• Out-of-Distribution Example Detection in Deep Neural Networks using Distance to Modelled Embedding
• Why Should we Combine Training and Post-Training Methods for Out-of-Distribution Detection?
• Testing for Typicality with Respect to an Ensemble of Learned Distributions
• Leveraging Uncertainty from Deep Learning for Trustworthy Materials Discovery Workflows
• MMF: A loss extension for feature learning in open set recognition
• Deep Unsupervised Image Anomaly Detection: An Information Theoretic Framework
• Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLU
• Data augmentation and pre-trained networks for extremely low data regimes unsupervised visual inspection
• Towards Interpretable and Learnable Risk Analysis for Entity Resolution
• Improving Compositionality of Neural Networks by Decoding Representations to Inputs
• Detection of Dataset Shifts in Learning-Enabled Cyber-Physical Systems using Variational Autoencoder for Regression
• Out-of-Distribution Detection for Dermoscopic Image Classification
• Unsupervised Learning of Multi-level Structures for Anomaly Detection
• Deep Learning and Open Set Malware Classification: A Survey
• Frustratingly Easy Uncertainty Estimation for Distribution Shift
• FROB: Few-shot ROBust Model for Classification and Out-of-Distribution Detection
• GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentation
• Multi-Class Anomaly Detection
• Towards Zero and Few-shot Knowledge-seeking Turn Detection in Task-orientated Dialogue Systems
• $f$-Cal: Calibrated aleatoric uncertainty estimation from neural networks for robot perception
• Evaluation of Out-of-Distribution Detection Performance of Self-Supervised Learning in a Controllable Environment
• Probabilistic Trust Intervals for Out of Distribution Detection
• Greedy Bayesian Posterior Approximation with Deep Ensembles