Do We Really Need to Learn Representations from In-domain Data for Outlier Detection?

Abstract

Unsupervised outlier detection, which predicts if a test sample is an outlier or not using only the information from unlabelled inlier data, is an important but challenging task. Recently, methods based on the two-stage framework achieve state-of-the-art performance on this task. The framework leverages self-supervised representation learning algorithms to train a feature extractor on inlier data, and applies a simple outlier detector in the feature space. In this paper, we explore the possibility of avoiding the high cost of training a distinct representation for each outlier detection task, and instead using a single pre-trained network as the universal feature extractor regardless of the source of in-domain data. In particular, we replace the task-specific feature extractor by one network pre-trained on ImageNet with a self-supervised loss. In experiments, we demonstrate competitive or better performance on a variety of outlier detection benchmarks compared with previous two-stage methods, suggesting that learning representations from in-domain data may be unnecessary for outlier detection.

References in corpus (25)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Improved Baselines with Momentum Contrastive Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• Robust Physical-World Attacks on Deep Learning Models
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Anomaly Detection using One-Class Neural Networks
• Assessing Generative Models via Precision and Recall
• TCLR: Temporal Contrastive Learning for Video Representation
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Generative Probabilistic Novelty Detection with Adversarial Autoencoders
• Classification-Based Anomaly Detection for General Data
• Implicit Generation and Generalization in Energy-Based Models
• Improving Reconstruction Autoencoder Out-of-distribution Detection with Mahalanobis Distance
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
• Contrastive Training for Improved Out-of-Distribution Detection
• SSD: A Unified Framework for Self-Supervised Outlier Detection
• Probabilistic Modeling of Deep Features for Out-of-Distribution and Adversarial Detection
• Perfect density models cannot guarantee anomaly detection
• Hybrid Discriminative-Generative Training via Contrastive Learning
• Confidence from Invariance to Image Transformations
• Hierarchical VAEs Know What They Don't Know