Being Bayesian about Categorical Probability
arXiv:2002.07965
Abstract
Neural networks utilize the softmax as a building block in classification tasks, which contains an overconfidence problem and lacks an uncertainty representation ability. As a Bayesian alternative to the softmax, we consider a random variable of a categorical probability over class labels. In this framework, the prior distribution explicitly models the presumed noise inherent in the observed label, which provides consistent gains in generalization performance in multiple challenging tasks. The proposed method inherits advantages of Bayesian approaches that achieve better uncertainty estimation and model calibration. Our method can be implemented as a plug-and-play loss function with negligible computational overhead compared to the softmax with the cross-entropy loss function.
arXiv commentICML 2020
References in corpus (16)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Distilling the Knowledge in a Neural Network
- On Calibration of Modern Neural Networks
- Weight Uncertainty in Neural Networks
- Momentum Contrast for Unsupervised Visual Representation Learning
- Evidential Deep Learning to Quantify Classification Uncertainty
- Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
- Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
- A Simple Baseline for Bayesian Uncertainty in Deep Learning
- Practical Deep Learning with Bayesian Principles
- Functional Variational Bayesian Neural Networks
- Stochastic Variational Deep Kernel Learning
- Noise Contrastive Priors for Functional Uncertainty
- Bayesian Inference for Large Scale Image Classification
- A Variational Dirichlet Framework for Out-of-Distribution Detection
- Quantifying Intrinsic Uncertainty in Classification via Deep Dirichlet Mixture Networks
Cited by in corpus (5)
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics
- Deep Evidential Regression
- Disturbing Target Values for Neural Network Regularization
- Diagnostic Uncertainty Calibration: Towards Reliable Machine Predictions in Medical Domain