Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts

Abstract

Accurate estimation of aleatoric and epistemic uncertainty is crucial to build safe and reliable systems. Traditional approaches, such as dropout and ensemble methods, estimate uncertainty by sampling probability predictions from different submodels, which leads to slow uncertainty estimation at inference time. Recent works address this drawback by directly predicting parameters of prior distributions over the probability predictions with a neural network. While this approach has demonstrated accurate uncertainty estimation, it requires defining arbitrary target parameters for in-distribution data and makes the unrealistic assumption that out-of-distribution (OOD) data is known at training time. In this work we propose the Posterior Network (PostNet), which uses Normalizing Flows to predict an individual closed-form posterior distribution over predicted probabilites for any input sample. The posterior distributions learned by PostNet accurately reflect uncertainty for in- and out-of-distribution data -- without requiring access to OOD data at training time. PostNet achieves state-of-the art results in OOD detection and in uncertainty calibration under dataset shifts.

arXiv commentNeurips 2020

References in corpus (10)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Deep Residual Learning for Image Recognition
• Conditional Image Generation with PixelCNN Decoders
• On Calibration of Modern Neural Networks
• Variational Inference with Normalizing Flows
• Evidential Deep Learning to Quantify Classification Uncertainty
• Practical Deep Learning with Bayesian Principles
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
• Uncertainty on Asynchronous Time Event Prediction

Cited by in corpus (7)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
• JuryGCN: Quantifying Jackknife Uncertainty on Graph Convolutional Networks
• Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detection
• Graceful Degradation and Related Fields
• On Out-of-distribution Detection with Energy-based Models
• Deep Classifiers with Label Noise Modeling and Distance Awareness