Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?

Abstract

Dirichlet-based uncertainty (DBU) models are a recent and promising class of uncertainty-aware models. DBU models predict the parameters of a Dirichlet distribution to provide fast, high-quality uncertainty estimates alongside with class predictions. In this work, we present the first large-scale, in-depth study of the robustness of DBU models under adversarial attacks. Our results suggest that uncertainty estimates of DBU models are not robust w.r.t. three important tasks: (1) indicating correctly and wrongly classified samples; (2) detecting adversarial examples; and (3) distinguishing between in-distribution (ID) and out-of-distribution (OOD) data. Additionally, we explore the first approaches to make DBU models more robust. While adversarial training has a minor effect, our median smoothing based approach significantly increases robustness of DBU models.

arXiv commentPublished at ICML 2021

References in corpus (15)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Weight Uncertainty in Neural Networks
• Adversarial Attacks on Neural Networks for Graph Data
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Certified Adversarial Robustness via Randomized Smoothing
• Evidential Deep Learning to Quantify Classification Uncertainty
• Practical Deep Learning with Bayesian Principles
• Single-Model Uncertainties for Deep Learning
• Certifiable Robustness to Graph Perturbations
• Prior Networks for Detection of Adversarial Attacks
• Collective Robustness Certificates: Exploiting Interdependence in Graph Neural Networks

Cited by in corpus (3)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification
• Provably Robust Detection of Out-of-distribution Data (almost) for free