Understanding Measures of Uncertainty for Adversarial Example Detection

Abstract

Measuring uncertainty is a promising technique for detecting adversarial examples, crafted inputs on which the model predicts an incorrect class with high confidence. But many measures of uncertainty exist, including predictive en- tropy and mutual information, each capturing different types of uncertainty. We study these measures, and shed light on why mutual information seems to be effective at the task of adversarial example detection. We highlight failure modes for MC dropout, a widely used approach for estimating uncertainty in deep models. This leads to an improved understanding of the drawbacks of current methods, and a proposal to improve the quality of uncertainty estimates using probabilistic model ensembles. We give illustrative experiments using MNIST to demonstrate the intuition underlying the different measures of uncertainty, as well as experiments on a real world Kaggle dogs vs cats classification dataset.

arXiv comment10 pages, 13 figures

References in corpus (5)

• Detecting Adversarial Samples from Artifacts
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• A Boundary Tilting Persepective on the Phenomenon of Adversarial Examples
• Boosting Adversarial Attacks with Momentum
• Adversarial Phenomenon in the Eyes of Bayesian Deep Learning

Cited by in corpus (55)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts
• Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Adversarial Attack Vulnerability of Medical Image Analysis Systems: Unexplored Factors
• POBA-GA: Perturbation Optimized Black-Box Adversarial Attacks via Genetic Algorithm
• QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results
• Practical Solutions for Machine Learning Safety in Autonomous Vehicles
• Discriminative out-of-distribution detection for semantic segmentation
• Uncertainty in Neural Networks: Approximately Bayesian Ensembling
• Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks
• Sufficient Conditions for Idealised Models to Have No Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks
• Deep Ensemble Bayesian Active Learning : Addressing the Mode Collapse issue in Monte Carlo dropout via Ensembles
• Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study
• The Limitations of Model Uncertainty in Adversarial Settings
• Improving black-box optimization in VAE latent space using decoder uncertainty
• Uncertainty Based Detection and Relabeling of Noisy Image Labels
• Uncertainty-Aware Machine Translation Evaluation
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
• Adversarial Training Versus Weight Decay
• Adversarial Attack for Uncertainty Estimation: Identifying Critical Regions in Neural Networks
• Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data
• Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks
• Dense Uncertainty Estimation
• Anomalous Example Detection in Deep Learning: A Survey
• Influence of uncertainty estimation techniques on false-positive reduction in liver lesion detection
• Continual Learning using a Bayesian Nonparametric Dictionary of Weight Factors
• LiBRe: A Practical Bayesian Approach to Adversarial Detection
• Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers
• Uncertainty aware audiovisual activity recognition using deep Bayesian variational inference
• Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain
• Capturing Label Characteristics in VAEs
• Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty
• Posterior Meta-Replay for Continual Learning
• Learning Multimodal VAEs through Mutual Supervision
• B-SCST: Bayesian Self-Critical Sequence Training for Image Captioning
• Robust quantum dots charge autotuning using neural network uncertainty
• Prior Networks for Detection of Adversarial Attacks
• Few Is Enough: Task-Augmented Active Meta-Learning for Brain Cell Classification
• DeDUCE: Generating Counterfactual Explanations Efficiently
• Scalable Multi-Class Bayesian Support Vector Machines for Structured and Unstructured Data
• Revisiting Explicit Regularization in Neural Networks for Well-Calibrated Predictive Uncertainty
• A Macro-Micro Weakly-supervised Framework for AS-OCT Tissue Segmentation
• Deep Latent Defence
• Probabilistic Safety for Bayesian Neural Networks
• Uncertainty based Class Activation Maps for Visual Question Answering
• Knowing More About Questions Can Help: Improving Calibration in Question Answering
• Deep Bayesian Image Set Classification: A Defence Approach against Adversarial Attacks
• HAWKEYE: Adversarial Example Detector for Deep Neural Networks
• The Effect of Prior Lipschitz Continuity on the Adversarial Robustness of Bayesian Neural Networks
• The Robust Semantic Segmentation UNCV2023 Challenge Results
• A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger's Adversarial Attacks
• Localized Uncertainty Attacks