Beyond Individual Input for Deep Anomaly Detection on Tabular Data

Abstract

Anomaly detection is vital in many domains, such as finance, healthcare, and cybersecurity. In this paper, we propose a novel deep anomaly detection method for tabular data that leverages Non-Parametric Transformers (NPTs), a model initially proposed for supervised tasks, to capture both feature-feature and sample-sample dependencies. In a reconstruction-based framework, we train an NPT to reconstruct masked features of normal samples. In a non-parametric fashion, we leverage the whole training set during inference and use the model's ability to reconstruct the masked features to generate an anomaly score. To the best of our knowledge, this is the first work to successfully combine feature-feature and sample-sample dependencies for anomaly detection on tabular datasets. Through extensive experiments on 31 benchmark tabular datasets, we demonstrate that our method achieves state-of-the-art performance, outperforming existing methods by 2.4% and 1.2% in terms of F1-score and AUROC, respectively. Our ablation study further proves that modeling both types of dependencies is crucial for anomaly detection on tabular data.

References in corpus (11)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Neural Machine Translation by Jointly Learning to Align and Translate
• Lookahead Optimizer: k steps forward, 1 step back
• SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training
• Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders
• Explainable Deep One-Class Classification
• Learning and Evaluating Representations for Deep One-class Classification
• A Mutual Information Maximization Perspective of Language Representation Learning
• Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning
• Neural Transformation Learning for Deep Anomaly Detection Beyond Images
• TracInAD: Measuring Influence for Anomaly Detection