A Flexible Framework for Anomaly Detection via Dimensionality Reduction
arXiv:1909.04060 · doi:10.1109/ISCMI47871.2019.9004400
Abstract
Anomaly detection is challenging, especially for large datasets in high dimensions. Here we explore a general anomaly detection framework based on dimensionality reduction and unsupervised clustering. We release DRAMA, a general python package that implements the general framework with a wide range of built-in options. We test DRAMA on a wide variety of simulated and real datasets, in up to 3000 dimensions, and find it robust and highly competitive with commonly-used anomaly detection algorithms, especially in high dimensions. The flexibility of the DRAMA framework allows for significant optimization once some examples of anomalies are available, making it ideal for online anomaly detection, active learning and highly unbalanced datasets.
arXiv comment6 pages
References in corpus (7)
- Scikit-learn: Machine Learning in Python
- Auto-Encoding Variational Bayes
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Why does deep and cheap learning work so well?
- Anomaly Detection in the Bitcoin System - A Network Perspective
- Less is More: Building Selective Anomaly Ensembles
- Bayesian Anomaly Detection and Classification