One Versus all for deep Neural Network Incertitude (OVNNI) quantification

Abstract

Deep neural networks (DNNs) are powerful learning models yet their results are not always reliable. This is due to the fact that modern DNNs are usually uncalibrated and we cannot characterize their epistemic uncertainty. In this work, we propose a new technique to quantify the epistemic uncertainty of data easily. This method consists in mixing the predictions of an ensemble of DNNs trained to classify One class vs All the other classes (OVA) with predictions from a standard DNN trained to perform All vs All (AVA) classification. On the one hand, the adjustment provided by the AVA DNN to the score of the base classifiers allows for a more fine-grained inter-class separation. On the other hand, the two types of classifiers enforce mutually their detection of out-of-distribution (OOD) samples, circumventing entirely the requirement of using such samples during training. Our method achieves state of the art performance in quantifying OOD data across multiple datasets and architectures while requiring little hyper-parameter tuning.

References in corpus (14)

• Weight Uncertainty in Neural Networks
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images
• Predictive Uncertainty Estimation via Prior Networks
• Deep Ensembles: A Loss Landscape Perspective
• Averaging Weights Leads to Wider Optima and Better Generalization
• A Simple Baseline for Bayesian Uncertainty in Deep Learning
• Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
• Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
• Outlier Exposure with Confidence Control for Out-of-Distribution Detection
• Metric Learning for Novelty and Anomaly Detection
• TRADI: Tracking deep neural network weight distributions for uncertainty estimation

Cited by in corpus (4)

• Dense outlier detection and open-set recognition based on training with noisy negative images
• Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks
• Learning a Discriminant Latent Space with Neural Discriminant Analysis
• Distributional Gaussian Process Layers for Outlier Detection in Image Segmentation