ESAD: End-to-end Deep Semi-supervised Anomaly Detection

Abstract

This paper explores semi-supervised anomaly detection, a more practical setting for anomaly detection where a small additional set of labeled samples are provided. We propose a new KL-divergence based objective function for semi-supervised anomaly detection, and show that two factors: the mutual information between the data and latent representations, and the entropy of latent representations, constitute an integral objective function for anomaly detection. To resolve the contradiction in simultaneously optimizing the two factors, we propose a novel encoder-decoder-encoder structure, with the first encoder focusing on optimizing the mutual information and the second encoder focusing on optimizing the entropy. The two encoders are enforced to share similar encoding with a consistent constraint on their latent representations. Extensive experiments have revealed that the proposed method significantly outperforms several state-of-the-arts on multiple benchmark datasets, including medical diagnosis and several classic anomaly detection benchmarks.

arXiv commentAccepted by BMVC 2021

References in corpus (15)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Learning deep representations by mutual information estimation and maximization
• Semi-Supervised Learning with Deep Generative Models
• Semi-Supervised Learning with Ladder Networks
• Semi-Supervised Learning with Generative Adversarial Networks
• A Comprehensive Survey of Data Mining-based Fraud Detection Research
• Deep Anomaly Detection Using Geometric Transformations
• Unsupervised and Semi-supervised Anomaly Detection with LSTM Neural Networks
• Toward Supervised Anomaly Detection
• Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series
• Coherence Pursuit: Fast, Simple, and Robust Principal Component Analysis
• Viral Pneumonia Screening on Chest X-ray Images Using Confidence-Aware Anomaly Detection
• OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representations
• Attribute Restoration Framework for Anomaly Detection
• Perceptual Image Anomaly Detection