The Hidden Uncertainty in a Neural Networks Activations

Abstract

The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data. This work investigates whether this distribution moreover correlates with a model's epistemic uncertainty, thus indicates its ability to generalise to novel inputs. We first empirically verify that epistemic uncertainty can be identified with the surprise, thus the negative log-likelihood, of observing a particular latent representation. Moreover, we demonstrate that the output-conditional distribution of hidden representations also allows quantifying aleatoric uncertainty via the entropy of the predictive distribution. We analyse epistemic and aleatoric uncertainty inferred from the representations of different layers and conclude that deeper layers lead to uncertainty with similar behaviour as established - but computationally more expensive - methods (e.g. deep ensembles). While our approach does not require modifying the training process, we follow prior work and experiment with an additional regularising loss that increases the information in the latent representations. We find that this leads to improved OOD detection of epistemic uncertainty at the cost of ambiguous calibration close to the data distribution. We verify our findings on both classification and regression models.

References in corpus (6)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• NICE: Non-linear Independent Components Estimation
• Deep Learning for Anomaly Detection: A Survey
• Guided Image Generation with Conditional Invertible Neural Networks
• Dropout Inference in Bayesian Neural Networks with Alpha-divergences
• Density of States Estimation for Out-of-Distribution Detection