Uncertainty Estimations by Softplus normalization in Bayesian Convolutional Neural Networks with Variational Inference

Abstract

We introduce a novel uncertainty estimation for classification tasks for Bayesian convolutional neural networks with variational inference. By normalizing the output of a Softplus function in the final layer, we estimate aleatoric and epistemic uncertainty in a coherent manner. The intractable posterior probability distributions over weights are inferred by Bayes by Backprop. Firstly, we demonstrate how this reliable variational inference method can serve as a fundamental construct for various network architectures. On multiple datasets in supervised learning settings (MNIST, CIFAR-10, CIFAR-100), this variational inference method achieves performances equivalent to frequentist inference in identical architectures, while the two desiderata, a measure for uncertainty and regularization are incorporated naturally. Secondly, we examine how our proposed measure for aleatoric and epistemic uncertainties is derived and validate it on the aforementioned datasets.

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Improving neural networks by preventing co-adaptation of feature detectors
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Weight Uncertainty in Neural Networks
• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• Variational Dropout and the Local Reparameterization Trick
• Auxiliary Deep Generative Models
• Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
• A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference
• Bayesian Recurrent Neural Networks
• Stochastic Backpropagation through Mixture Density Distributions
• Variance Networks: When Expectation Does Not Meet Your Expectations

Cited by in corpus (13)

• A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference
• Bayesian Layers: A Module for Neural Network Uncertainty
• Parameters Estimation for the Cosmic Microwave Background with Bayesian Neural Networks
• Dim but not entirely dark: Extracting the Galactic Center Excess' source-count distribution with neural nets
• Uncertainty-guided Continual Learning with Bayesian Neural Networks
• ProbAct: A Probabilistic Activation Function for Deep Neural Networks
• Bayesian Convolutional Neural Networks for Limited Data Hyperspectral Remote Sensing Image Classification
• Why have a Unified Predictive Uncertainty? Disentangling it using Deep Split Ensembles
• On the Robustness of Monte Carlo Dropout Trained with Noisy Labels
• Explanation and Use of Uncertainty Quantified by Bayesian Neural Network Classifiers for Breast Histopathology Images
• Multi Agent System for Machine Learning Under Uncertainty in Cyber Physical Manufacturing System
• Bayesian deep learning of affordances from RGB images
• An Uncertainty-aware Hierarchical Probabilistic Network for Early Prediction, Quantification and Segmentation of Pulmonary Tumour Growth