Provably Robust Detection of Out-of-distribution Data (almost) for free

Abstract

The application of machine learning in safety-critical systems requires a reliable assessment of uncertainty. However, deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. Even if trained to be non-confident on OOD data, one can still adversarially manipulate OOD data so that the classifier again assigns high confidence to the manipulated samples. We show that two previously published defenses can be broken by better adapted attacks, highlighting the importance of robustness guarantees around OOD data. Since the existing method for this task is hard to train and significantly limits accuracy, we construct a classifier that can simultaneously achieve provably adversarially robust OOD detection and high clean accuracy. Moreover, by slightly modifying the classifier's architecture our method provably avoids the asymptotic overconfidence problem of standard neural networks. We provide code for all our experiments.

References in corpus (15)

• On Calibration of Modern Neural Networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Fine-Grained Visual Classification of Aircraft
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Evidential Deep Learning to Quantify Classification Uncertainty
• Deep Anomaly Detection with Outlier Exposure
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• On Adaptive Attacks to Adversarial Example Defenses
• Outlier Exposure with Confidence Control for Out-of-Distribution Detection
• Certifiably Adversarially Robust Detection of Out-of-Distribution Data
• Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
• Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?
• An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence

Cited by in corpus (3)

• A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges
• DOODLER: Determining Out-Of-Distribution Likelihood from Encoder Reconstructions
• FROB: Few-shot ROBust Model for Classification and Out-of-Distribution Detection