Energy-based Out-of-distribution Detection

Abstract

Determining whether inputs are out-of-distribution (OOD) is an essential building block for safely deploying machine learning models in the open world. However, previous methods relying on the softmax confidence score suffer from overconfident posterior distributions for OOD data. We propose a unified framework for OOD detection that uses an energy score. We show that energy scores better distinguish in- and out-of-distribution samples than the traditional approach using the softmax scores. Unlike softmax confidence scores, energy scores are theoretically aligned with the probability density of the inputs and are less susceptible to the overconfidence issue. Within this framework, energy can be flexibly used as a scoring function for any pre-trained neural classifier as well as a trainable cost function to shape the energy surface explicitly for OOD detection. On a CIFAR-10 pre-trained WideResNet, using the energy score reduces the average FPR (at TPR 95%) by 18.03% compared to the softmax confidence score. With energy-based training, our method outperforms the state-of-the-art on common benchmarks.

References in corpus (17)

• Auto-Encoding Variational Bayes
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Conditional Image Generation with PixelCNN Decoders
• Wide Residual Networks
• SGDR: Stochastic Gradient Descent with Warm Restarts
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Density estimation using Real NVP
• Deep Anomaly Detection with Outlier Exposure
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking
• A Theory of Generative ConvNet
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Implicit Generation and Generalization in Energy-Based Models
• Discriminative out-of-distribution detection for semantic segmentation
• Confidence estimation in Deep Neural networks via density modelling

Cited by in corpus (34)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Contrastive Out-of-Distribution Detection for Pretrained Transformers
• Fine-grained TLS services classification with reject option
• Iterative Human and Automated Identification of Wildlife Images
• Energy-based Detection of Adverse Weather Effects in LiDAR Data
• Entropic Out-of-Distribution Detection: Seamless Detection of Unknown Examples
• Learning Discriminative Representations and Decision Boundaries for Open Intent Detection
• A modular U-Net for automated segmentation of X-ray tomography images in composite materials
• Perfect density models cannot guarantee anomaly detection
• Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness
• Maximizing Discrimination Capability of Knowledge Distillation with Energy Function
• Towards Trustworthy Dataset Distillation
• Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning
• Causality and Independence Enhancement for Biased Node Classification
• No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets
• DeepLens: Interactive Out-of-distribution Data Detection in NLP Models
• Probing the Purview of Neural Networks via Gradient Analysis
• Open-World Lifelong Graph Learning
• Elsa: Energy-based learning for semi-supervised anomaly detection
• ReSmooth: Detecting and Utilizing OOD Samples when Training with Data Augmentation
• Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection
• Conditioning Latent-Space Clusters for Real-World Anomaly Classification
• Exploring Vicinal Risk Minimization for Lightweight Out-of-Distribution Detection
• Energy-Based Pseudo-Label Refining for Source-free Domain Adaptation
• Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data
• Deep Out-of-Distribution Uncertainty Quantification via Weight Entropy Maximization
• A Multi-Scale Framework for Out-of-Distribution Detection in Dermoscopic Images
• Energy Aligning for Biased Models
• Detecting Out-of-distribution Objects Using Neuron Activation Patterns
• The Robust Semantic Segmentation UNCV2023 Challenge Results
• Natural Attribute-based Shift Detection
• Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty
• Toward a Realistic Benchmark for Out-of-Distribution Detection
• On the detection of Out-Of-Distribution samples in Multiple Instance Learning