Inhibited Softmax for Uncertainty Estimation in Neural Networks
arXiv:1810.01861
Abstract
We present a new method for uncertainty estimation and out-of-distribution detection in neural networks with softmax output. We extend softmax layer with an additional constant input. The corresponding additional output is able to represent the uncertainty of the network. The proposed method requires neither additional parameters nor multiple forward passes nor input preprocessing nor out-of-distribution datasets. We show that our method performs comparably to more computationally expensive methods and outperforms baselines on our experiments from image recognition and sentiment analysis domains.
References in corpus (12)
- ADADELTA: An Adaptive Learning Rate Method
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- On Calibration of Modern Neural Networks
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
- Bayesian Active Learning for Classification and Preference Learning
- Learning Confidence for Out-of-Distribution Detection in Neural Networks
- Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
- Bayesian Dark Knowledge
- Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
- Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam
- Adversarial Distillation of Bayesian Neural Network Posteriors
Cited by in corpus (4)
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- AI Security for Geoscience and Remote Sensing: Challenges and Future Trends
- Learning Uncertainty For Safety-Oriented Semantic Segmentation In Autonomous Driving
- DS-UI: Dual-Supervised Mixture of Gaussian Mixture Models for Uncertainty Inference