Deep Isolation Forest for Anomaly Detection

Abstract

Isolation forest (iForest) has been emerging as arguably the most popular anomaly detector in recent years due to its general effectiveness across different benchmarks and strong scalability. Nevertheless, its linear axis-parallel isolation method often leads to (i) failure in detecting hard anomalies that are difficult to isolate in high-dimensional/non-linear-separable data space, and (ii) notorious algorithmic bias that assigns unexpectedly lower anomaly scores to artefact regions. These issues contribute to high false negative errors. Several iForest extensions are introduced, but they essentially still employ shallow, linear data partition, restricting their power in isolating true anomalies. Therefore, this paper proposes deep isolation forest. We introduce a new representation scheme that utilises casually initialised neural networks to map original data into random representation ensembles, where random axis-parallel cuts are subsequently applied to perform the data partition. This representation scheme facilitates high freedom of the partition in the original data space (equivalent to non-linear partition on subspaces of varying sizes), encouraging a unique synergy between random representations and random partition-based isolation. Extensive experiments show that our model achieves significant improvement over state-of-the-art isolation-based methods and deep detectors on tabular, graph and time series datasets; our model also inherits desired scalability from iForest.

Accepted by IEEE Transactions on Knowledge and Data Engineering (TKDE)

References in corpus (7)

• ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions
• Toward Deep Supervised Anomaly Detection: Reinforcement Learning from Partially Labeled Anomaly Data
• Deep Graph-level Anomaly Detection by Glocal Knowledge Distillation
• Sintel: A Machine Learning Framework to Extract Insights from Signals
• Diversity Matters When Learning From Ensembles
• Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
• DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation

Cited by in corpus (8)

• DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series
• Anomaly Detection Based on Isolation Mechanisms: A Survey
• Learn to Rank Risky Investors: A Case Study of Predicting Retail Traders' Behaviour and Profitability
• TriP-LLM: A Tri-Branch Patch-wise Large Language Model Framework for Time-Series Anomaly Detection
• Label-Informed Outlier Detection Based on Granule Density
• Mitigating Long-Tailed Anomaly Score Distributions with Importance-Weighted Loss
• Critical Review for One-class Classification: recent advances and the reality behind them
• Flexible Simulation Based Inference for Galaxy Photometric Fitting with Synthesizer