Deep Unsupervised Image Anomaly Detection: An Information Theoretic Framework

Abstract

Surrogate task based methods have recently shown great promise for unsupervised image anomaly detection. However, there is no guarantee that the surrogate tasks share the consistent optimization direction with anomaly detection. In this paper, we return to a direct objective function for anomaly detection with information theory, which maximizes the distance between normal and anomalous data in terms of the joint distribution of images and their representation. Unfortunately, this objective function is not directly optimizable under the unsupervised setting where no anomalous data is provided during training. Through mathematical analysis of the above objective function, we manage to decompose it into four components. In order to optimize in an unsupervised fashion, we show that, under the assumption that distribution of the normal and anomalous data are separable in the latent space, its lower bound can be considered as a function which weights the trade-off between mutual information and entropy. This objective function is able to explain why the surrogate task based methods are effective for anomaly detection and further point out the potential direction of improvement. Based on this object function we introduce a novel information theoretic framework for unsupervised image anomaly detection. Extensive experiments have demonstrated that the proposed framework significantly outperforms several state-of-the-arts on multiple benchmark data sets.

References in corpus (12)

• A Simple Framework for Contrastive Learning of Visual Representations
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• The information bottleneck method
• Learning deep representations by mutual information estimation and maximization
• Deep Learning for Anomaly Detection: A Survey
• Adversarially Learned Inference
• Deep Anomaly Detection with Outlier Exposure
• Regularizing Neural Networks by Penalizing Confident Output Distributions
• Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series
• On Variational Bounds of Mutual Information
• Coherence Pursuit: Fast, Simple, and Robust Principal Component Analysis
• Attribute Restoration Framework for Anomaly Detection