Uncertainty Surrogates for Deep Learning

Abstract

In this paper we introduce a novel way of estimating prediction uncertainty in deep networks through the use of uncertainty surrogates. These surrogates are features of the penultimate layer of a deep network that are forced to match predefined patterns. The patterns themselves can be, among other possibilities, a known visual symbol. We show how our approach can be used for estimating uncertainty in prediction and out-of-distribution detection. Additionally, the surrogates allow for interpretability of the ability of the deep network to learn and at the same time lend robustness against adversarial attacks. Despite its simplicity, our approach is superior to the state-of-the-art approaches on standard metrics as well as computational efficiency and ease of implementation. A wide range of experiments are performed on standard datasets to prove the efficacy of our approach.

References in corpus (14)

• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• Deep and Confident Prediction for Time Series at Uber
• Exploration by Random Network Distillation
• High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach
• BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning
• Single-Model Uncertainties for Deep Learning
• Hydra: Preserving Ensemble Diversity for Model Distillation
• Training independent subnetworks for robust prediction
• Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?
• Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel
• Modelling heterogeneous distributions with an Uncountable Mixture of Asymmetric Laplacians
• Prior Networks for Detection of Adversarial Attacks