Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model

Abstract

Uncertainty estimation has been extensively studied in recent literature, which can usually be classified as aleatoric uncertainty and epistemic uncertainty. In current aleatoric uncertainty estimation frameworks, it is often neglected that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. Since the oracle model is inaccessible in most cases, we propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Further, we show a trivial solution in the dual-head based heteroscedastic aleatoric uncertainty estimation framework and introduce a new uncertainty consistency loss to avoid it. For epistemic uncertainty estimation, we argue that the internal variable in a conditional latent variable model is another source of epistemic uncertainty to model the predictive distribution and explore the limited knowledge about the hidden true model. We validate our observation on a dense prediction task, i.e., camouflaged object detection. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.

References in corpus (20)

• Auto-Encoding Variational Bayes
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• On Calibration of Modern Neural Networks
• Anabranch Network for Camouflaged Object Segmentation
• Adversarial Learning for Semi-Supervised Semantic Segmentation
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation
• Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
• Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
• Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning
• What Are Bayesian Neural Network Posteriors Really Like?
• BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning
• Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization
• SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates
• On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty
• Alternating Back-Propagation for Generator Network
• Explaining Bayesian Neural Networks
• A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise
• Uncertainty-Aware Deep Calibrated Salient Object Detection
• Deep Probabilistic Ensembles: Approximate Variational Inference through KL Regularization