Learning Confidence for Out-of-Distribution Detection in Neural Networks
arXiv:1802.04865
Abstract
Modern neural networks are very powerful predictive models, but they are often incapable of recognizing when their predictions may be wrong. Closely related to this is the task of out-of-distribution detection, where a network must determine whether or not an input is outside of the set on which it is expected to safely perform. To jointly address these issues, we propose a method of learning confidence estimates for neural networks that is simple to implement and produces intuitively interpretable outputs. We demonstrate that on the task of out-of-distribution detection, our technique surpasses recently proposed techniques which construct confidence based on the network's output distribution, without requiring any additional labels or access to out-of-distribution examples. Additionally, we address the problem of calibrating out-of-distribution detectors, where we demonstrate that misclassified in-distribution examples can be used as a proxy for out-of-distribution examples.
References in corpus (3)
Cited by in corpus (107)
- Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
- Energy-based Out-of-distribution Detection
- Generalized Out-of-Distribution Detection: A Survey
- Scaling Out-of-Distribution Detection for Real-World Settings
- The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
- Generative Probabilistic Novelty Detection with Adversarial Autoencoders
- Practical Deep Learning with Bayesian Principles
- Addressing Failure Prediction by Learning Model Confidence
- The Conditional Entropy Bottleneck
- Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder
- SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation
- Large-Scale Long-Tailed Recognition in an Open World
- Leveraging Uncertainty Estimates for Predicting Segmentation Quality
- Contrastive Training for Improved Out-of-Distribution Detection
- Towards increased trustworthiness of deep learning segmentation methods on cardiac MRI
- Practical Solutions for Machine Learning Safety in Autonomous Vehicles
- Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
- Out-of-distribution Detection in Classifiers via Generation
- SSD: A Unified Framework for Self-Supervised Outlier Detection
- Detecting Out-of-Distribution Inputs in Deep Neural Networks Using an Early-Layer Output
- Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks
- Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
- Did You Miss the Sign? A False Negative Alarm System for Traffic Sign Detectors
- Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teacher
- Entropic Out-of-Distribution Detection: Seamless Detection of Unknown Examples
- Dense outlier detection and open-set recognition based on training with noisy negative images
- Towards out of distribution generalization for problems in mechanics
- AI Research Considerations for Human Existential Safety (ARCHES)
- OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers
- Inhibited Softmax for Uncertainty Estimation in Neural Networks
- Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices
- Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference
- Unbiased Mean Teacher for Cross-domain Object Detection
- On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation
- Corner Cases for Visual Perception in Automated Driving: Some Guidance on Detection Approaches
- Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection
- Unsupervised Out-of-Distribution Detection with Batch Normalization
- Distill-SODA: Distilling Self-Supervised Vision Transformer for Source-Free Open-Set Domain Adaptation in Computational Pathology
- AutoOD: Automated Outlier Detection via Curiosity-guided Search and Self-imitation Learning
- EigenRank by Committee: A Data Subset Selection and Failure Prediction paradigm for Robust Deep Learning based Medical Image Segmentation
- Density of States Estimation for Out-of-Distribution Detection
- TRUST-LAPSE: An Explainable and Actionable Mistrust Scoring Framework for Model Monitoring
- Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity
- A Variational Dirichlet Framework for Out-of-Distribution Detection
- Detecting Adversarial Examples and Other Misclassifications in Neural Networks by Introspection
- SoK: Machine Learning Governance
- Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation
- Hybrid Open-set Segmentation with Synthetic Negative Data
- A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks
- DOCTOR: A Simple Method for Detecting Misclassification Errors
- ReMix: Calibrated Resampling for Class Imbalance in Deep learning
- Label-Free Model Evaluation with Semi-Structured Dataset Representations
- Out-of-Distribution Detection for Generalized Zero-Shot Action Recognition
- Uncertainty-Aware Segmentation Quality Prediction via Deep Learning Bayesian Modeling: Comprehensive Evaluation and Interpretation on Skin Cancer and Liver Segmentation
- OneFlow: One-class flow for anomaly detection based on a minimal volume region
- Modeling Uncertainty by Learning a Hierarchy of Deep Neural Connections
- Revisiting the Evaluation of Uncertainty Estimation and Its Application to Explore Model Complexity-Uncertainty Trade-Off
- Quantifying Intrinsic Uncertainty in Classification via Deep Dirichlet Mixture Networks
- MetaFusion: Controlled False-Negative Reduction of Minority Classes in Semantic Segmentation
- Probing the Purview of Neural Networks via Gradient Analysis
- Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
- On the Role of Dataset Quality and Heterogeneity in Model Confidence
- Efficacy of Pixel-Level OOD Detection for Semantic Segmentation
- DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation
- SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detection
- Deep Learning and Traffic Classification: Lessons learned from a commercial-grade dataset with hundreds of encrypted and zero-day applications
- InFlow: Robust outlier detection utilizing Normalizing Flows
- Folden: -Fold Ensemble for Out-Of-Distribution Detection
- Modeling Token-level Uncertainty to Learn Unknown Concepts in SLU via Calibrated Dirichlet Prior RNN
- Out-of-Distribution Detection for Skin Lesion Images with Deep Isolation Forest
- A t-distribution based operator for enhancing out of distribution robustness of neural network classifiers
- Bayesian OOD detection with aleatoric uncertainty and outlier exposure
- Learning with Out-of-Distribution Data for Audio Classification
- Interactive Medical Image Segmentation with Self-Adaptive Confidence Calibration
- SCOUT: Self-aware Discriminant Counterfactual Explanations
- Model selection for deep audio source separation via clustering analysis
- Toward Metrics for Differentiating Out-of-Distribution Sets
- Bridging In- and Out-of-distribution Samples for Their Better Discriminability
- Safety Case Templates for Autonomous Systems
- Topological Uncertainty: Monitoring trained neural networks through persistence of activation graphs
- Efficient Action Recognition Using Confidence Distillation
- UNO: Uncertainty-aware Noisy-Or Multimodal Fusion for Unanticipated Input Degradation
- Utilizing Network Properties to Detect Erroneous Inputs
- Deep Out-of-Distribution Uncertainty Quantification via Weight Entropy Maximization
- Thought Flow Nets: From Single Predictions to Trains of Model Thought
- MOOD: Multi-level Out-of-distribution Detection
- Unsupervised Class-Incremental Learning Through Confusion
- Robust Out-of-Distribution Detection on Deep Probabilistic Generative Models
- Controllable Confidence-Based Image Denoising
- A Survey on Assessing the Generalization Envelope of Deep Neural Networks: Predictive Uncertainty, Out-of-distribution and Adversarial Samples
- DeepStreamCE: A Streaming Approach to Concept Evolution Detection in Deep Neural Networks
- Power Pooling Operators and Confidence Learning for Semi-Supervised Sound Event Detection
- Pixel Invisibility: Detecting Objects Invisible in Color Images
- DeepConsensus: using the consensus of features from multiple layers to attain robust image classification
- Multi-class Novelty Detection Using Mix-up Technique
- The Effect of Optimization Methods on the Robustness of Out-of-Distribution Detection Approaches
- Know Where To Drop Your Weights: Towards Faster Uncertainty Estimation
- GalilAI: Out-of-Task Distribution Detection using Causal Active Experimentation for Safe Transfer RL
- A novel network training approach for open set image recognition
- Improve Uncertainty Estimation for Unknown Classes in Bayesian Neural Networks with Semi-Supervised /One Set Classification
- Multi-Class Anomaly Detection
- GIM: Gaussian Isolation Machines
- AUC Optimization for Robust Small-footprint Keyword Spotting with Limited Training Data
- Image-based OoD-Detector Principles on Graph-based Input Data in Human Action Recognition
- Toward Auto-evaluation with Confidence-based Category Relation-aware Regression
- Closing the Gap: Achieving Better Accuracy-Robustness Tradeoffs against Query-Based Attacks
- Self-Checking Deep Neural Networks in Deployment