Uncertainty Estimation Using a Single Deep Deterministic Neural Network

Abstract

We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. Our approach, deterministic uncertainty quantification (DUQ), builds upon ideas of RBF networks. We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models. By enforcing detectability of changes in the input using a gradient penalty, we are able to reliably detect out of distribution data. Our uncertainty quantification scales well to large datasets, and using a single model, we improve upon or match Deep Ensembles in out of distribution detection on notable difficult dataset pairs such as FashionMNIST vs. MNIST, and CIFAR-10 vs. SVHN.

References in corpus (16)

• Scikit-learn: Machine Learning in Python
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Neural Discrete Representation Learning
• NICE: Non-linear Independent Components Estimation
• Weight Uncertainty in Neural Networks
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Bayesian Active Learning for Classification and Preference Learning
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Understanding Measures of Uncertainty for Adversarial Example Detection
• On the regularization of Wasserstein GANs
• Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
• A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks
• Robust Learning with Jacobian Regularization
• Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning

Cited by in corpus (42)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Generalized Out-of-Distribution Detection: A Survey
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
• The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
• Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
• Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions
• On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty
• Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena
• Bayesian Deep Ensembles via the Neural Tangent Kernel
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
• CertainNet: Sampling-free Uncertainty Estimation for Object Detection
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
• SAMBA: Safe Model-Based & Active Reinforcement Learning
• Corner Cases for Visual Perception in Automated Driving: Some Guidance on Detection Approaches
• Lightweight Regression Model with Prediction Interval Estimation for Computer Vision-based Winter Road Surface Condition Monitoring
• Adversarial Attack for Uncertainty Estimation: Identifying Critical Regions in Neural Networks
• Reliable Graph Neural Networks for Drug Discovery Under Distributional Shift
• Stochastic-YOLO: Efficient Probabilistic Object Detection under Dataset Shifts
• JuryGCN: Quantifying Jackknife Uncertainty on Graph Convolutional Networks
• DS-UI: Dual-Supervised Mixture of Gaussian Mixture Models for Uncertainty Inference
• DUDES: Deep Uncertainty Distillation using Ensembles for Semantic Segmentation
• Toward Reliable Human Pose Forecasting with Uncertainty
• Confidence Aware Neural Networks for Skin Cancer Detection
• Influence of uncertainty estimation techniques on false-positive reduction in liver lesion detection
• Segmenting Known Objects and Unseen Unknowns without Prior Knowledge
• Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
• Deep Anti-Regularized Ensembles provide reliable out-of-distribution uncertainty quantification
• Distance Matters For Improving Performance Estimation Under Covariate Shift
• Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data
• Trustworthy Long-Tailed Classification
• An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
• Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures
• DeDUCE: Generating Counterfactual Explanations Efficiently
• The Compact Support Neural Network
• Distributional Gaussian Process Layers for Outlier Detection in Image Segmentation
• Bayesian Uncertainty and Expected Gradient Length -- Regression: Two Sides Of The Same Coin?
• Deep Out-of-Distribution Uncertainty Quantification via Weight Entropy Maximization
• FROB: Few-shot ROBust Model for Classification and Out-of-Distribution Detection
• Greedy Bayesian Posterior Approximation with Deep Ensembles
• Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model