Why is the Mahalanobis Distance Effective for Anomaly Detection?

Abstract

The Mahalanobis distance-based confidence score, a recently proposed anomaly detection method for pre-trained neural classifiers, achieves state-of-the-art performance on both out-of-distribution (OoD) and adversarial examples detection. This work analyzes why this method exhibits such strong performance in practical settings while imposing an implausible assumption; namely, that class conditional distributions of pre-trained features have tied covariance. Although the Mahalanobis distance-based method is claimed to be motivated by classification prediction confidence, we find that its superior performance stems from information not useful for classification. This suggests that the reason the Mahalanobis confidence score works so well is mistaken, and makes use of different information from ODIN, another popular OoD detection method based on prediction confidence. This perspective motivates us to combine these two methods, and the combined detector exhibits improved performance and robustness. These findings provide insight into the behavior of neural classifiers in response to anomalous inputs.

References in corpus (17)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Intriguing properties of neural networks
• Prototypical Networks for Few-shot Learning
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• EMNIST: an extension of MNIST to handwritten letters
• Deep Anomaly Detection with Outlier Exposure
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• Detecting Adversarial Samples from Artifacts
• On the (Statistical) Detection of Adversarial Examples
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Distance-based Confidence Score for Neural Network Classifiers
• Improving Reconstruction Autoencoder Out-of-distribution Detection with Mahalanobis Distance
• Probabilistic Modeling of Deep Features for Out-of-Distribution and Adversarial Detection
• Accurate Layerwise Interpretable Competence Estimation
• Interpretable Disentanglement of Neural Networks by Extracting Class-Specific Subnetwork

Cited by in corpus (7)

• A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
• Contrastive Training for Improved Out-of-Distribution Detection
• Feature Space Singularity for Out-of-Distribution Detection
• Transfer Learning Gaussian Anomaly Detection by Fine-tuning Representations
• Graceful Degradation and Related Fields
• A Distance-based Anomaly Detection Framework for Deep Reinforcement Learning
• Data augmentation and pre-trained networks for extremely low data regimes unsupervised visual inspection