Detecting Adversarial Examples and Other Misclassifications in Neural Networks by Introspection

Abstract

Despite having excellent performances for a wide variety of tasks, modern neural networks are unable to provide a reliable confidence value allowing to detect misclassifications. This limitation is at the heart of what is known as an adversarial example, where the network provides a wrong prediction associated with a strong confidence to a slightly modified image. Moreover, this overconfidence issue has also been observed for regular errors and out-of-distribution data. We tackle this problem by what we call introspection, i.e. using the information provided by the logits of an already pretrained neural network. We show that by training a simple 3-layers neural network on top of the logit activations, we are able to detect misclassifications at a competitive level.

5 pages, 2 figures, Presented at the ICML 2019 Workshop on Uncertainty and Robustness in Deep Learning

References in corpus (5)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• HopSkipJumpAttack: A Query-Efficient Decision-Based Attack

Cited by in corpus (5)

• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Anomalous Example Detection in Deep Learning: A Survey
• Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain
• Using Anomaly Feature Vectors for Detecting, Classifying and Warning of Outlier Adversarial Examples
• MixDefense: A Defense-in-Depth Framework for Adversarial Example Detection Based on Statistical and Semantic Analysis