Deep Structured Energy Based Models for Anomaly Detection

Abstract

In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures. We propose deep structured energy based models (DSEBMs), where the energy function is the output of a deterministic deep neural network with structure. We develop novel model architectures to integrate EBMs with different types of data such as static data, sequential data, and spatial data, and apply appropriate model architectures to adapt to the data structure. Our training algorithm is built upon the recent development of score matching \cite{sm}, which connects an EBM with a regularized autoencoder, eliminating the need for complicated sampling method. Statistically sound decision criterion can be derived for anomaly detection purpose from the perspective of the energy landscape of the data distribution. We investigate two decision criteria for performing anomaly detection: the energy score and the reconstruction error. Extensive empirical studies on benchmark tasks demonstrate that our proposed model consistently matches or outperforms all the competing methods.

To appear in ICML 2016

References in corpus (1)

• Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription

Cited by in corpus (47)

• Deep Learning for Anomaly Detection: A Survey
• Deep Anomaly Detection Using Geometric Transformations
• Anomaly Detection for a Water Treatment System Using Unsupervised Machine Learning
• Generalized Out-of-Distribution Detection: A Survey
• Anomaly Detection in Medical Imaging with Deep Perceptual Autoencoders
• Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection
• A Survey on Unsupervised Anomaly Detection Algorithms for Industrial Images
• Towards Explaining Anomalies: A Deep Taylor Decomposition of One-Class Models
• A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data
• Learning and Evaluating Representations for Deep One-class Classification
• Detection of Anomalies in Large Scale Accounting Data using Deep Autoencoder Networks
• MSTREAM: Fast Anomaly Detection in Multi-Aspect Streams
• Maximum Entropy Generators for Energy-Based Models
• Machine Learning in NextG Networks via Generative Adversarial Networks
• Adaptive Model Pooling for Online Deep Anomaly Detection from a Complex Evolving Data Stream
• Learning Memory-guided Normality for Anomaly Detection
• Anomaly Detection based on Zero-Shot Outlier Synthesis and Hierarchical Feature Distillation
• Unsupervised Outlier Detection using Memory and Contrastive Learning
• End-to-End Learning for Structured Prediction Energy Networks
• Domain Adaptive Transfer Learning for Fault Diagnosis
• Generative Adversarial Networks as Variational Training of Energy Based Models
• Puzzle-AE: Novelty Detection in Images through Solving Puzzles
• Exploring Unsupervised Anomaly Detection with Quantum Boltzmann Machines in Fraud Detection
• Rethinking Assumptions in Deep Anomaly Detection
• Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders
• Estimation of Dimensions Contributing to Detected Anomalies with Variational Autoencoders
• Fence GAN: Towards Better Anomaly Detection
• OneFlow: One-class flow for anomaly detection based on a minimal volume region
• Detecting Anomalous Faces with 'No Peeking' Autoencoders
• Outlier classification using Autoencoders: application for fluctuation driven flows in fusion plasmas
• Unsupervised Cyberbullying Detection via Time-Informed Gaussian Mixture Model
• Self-Supervised Representation Learning for Visual Anomaly Detection
• Flow Contrastive Estimation of Energy-Based Models
• Deep Variational Semi-Supervised Novelty Detection
• Autoencoding Binary Classifiers for Supervised Anomaly Detection
• Arbitrary Discrete Sequence Anomaly Detection with Zero Boundary LSTM
• Graph Regularized Autoencoder and its Application in Unsupervised Anomaly Detection
• On Out-of-distribution Detection with Energy-based Models
• Medical Knowledge-Guided Deep Learning for Imbalanced Medical Image Classification
• Acoustic anomaly detection via latent regularized gaussian mixture generative adversarial networks
• Multilevel Anomaly Detection for Mixed Data
• Deep Structured Cross-Modal Anomaly Detection
• Energy-Based Anomaly Detection and Localization
• Flow-based SVDD for anomaly detection
• Transfer Learning from an Auxiliary Discriminative Task for Unsupervised Anomaly Detection
• Hierarchical Models: Intrinsic Separability in High Dimensions
• Self-Organizing Map assisted Deep Autoencoding Gaussian Mixture Model for Intrusion Detection