Confidence estimation in Deep Neural networks via density modelling

Abstract

State-of-the-art Deep Neural Networks can be easily fooled into providing incorrect high-confidence predictions for images with small amounts of adversarial noise. Does this expose a flaw with deep neural networks, or do we simply need a better way to estimate confidence? In this paper we consider the problem of accurately estimating predictive confidence. We formulate this problem as that of density modelling, and show how traditional methods such as softmax produce poor estimates. To address this issue, we propose a novel confidence measure based on density modelling approaches. We test these measures on images distorted by blur, JPEG compression, random noise and adversarial noise. Experiments show that our confidence measure consistently shows reduced confidence scores in the presence of such distortions - a property which softmax often lacks.

ICME 2017

References in corpus (2)

• Universal adversarial perturbations
• Improving the Robustness of Deep Neural Networks via Stability Training

Cited by in corpus (10)

• Energy-based Out-of-distribution Detection
• Deep Confidence: A Computationally Efficient Framework for Calculating Reliable Errors for Deep Neural Networks
• Robust Out-of-distribution Detection for Neural Networks
• Entropic Out-of-Distribution Detection: Seamless Detection of Unknown Examples
• Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity
• ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining
• MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space
• Progressive Neural Networks for Image Classification
• Model selection for deep audio source separation via clustering analysis
• Enhanced Isotropy Maximization Loss: Seamless and High-Performance Out-of-Distribution Detection Simply Replacing the SoftMax Loss