Unsupervised Out-of-Distribution Detection with Batch Normalization

Abstract

Likelihood from a generative model is a natural statistic for detecting out-of-distribution (OoD) samples. However, generative models have been shown to assign higher likelihood to OoD samples compared to ones from the training distribution, preventing simple threshold-based detection rules. We demonstrate that OoD detection fails even when using more sophisticated statistics based on the likelihoods of individual samples. To address these issues, we propose a new method that leverages batch normalization. We argue that batch normalization for generative models challenges the traditional i.i.d. data assumption and changes the corresponding maximum likelihood objective. Based on this insight, we propose to exploit in-batch dependencies for OoD detection. Empirical results suggest that this leads to more robust detection for high-dimensional images.

References in corpus (21)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• NICE: Non-linear Independent Components Estimation
• End-to-end Optimized Image Compression
• Density estimation using Real NVP
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• A note on the evaluation of generative models
• Deep Anomaly Detection with Outlier Exposure
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• How Does Batch Normalization Help Optimization?
• Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Generative Probabilistic Novelty Detection with Adversarial Autoencoders
• Sylvester Normalizing Flows for Variational Inference
• A Kernel Test of Goodness of Fit
• EX2: Exploration with Exemplar Models for Deep Reinforcement Learning
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
• Novelty Detection with GAN
• Are generative deep models for novelty detection truly better?
• Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization
• Group Anomaly Detection using Deep Generative Models

Cited by in corpus (7)

• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
• Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder
• Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices
• Anomalous Example Detection in Deep Learning: A Survey
• Bigeminal Priors Variational auto-encoder
• DOI: Divergence-based Out-of-Distribution Indicators via Deep Generative Models
• Knowing More About Questions Can Help: Improving Calibration in Question Answering