On the Out-of-distribution Generalization of Probabilistic Image Modelling
arXiv:2109.02639
Abstract
Out-of-distribution (OOD) detection and lossless compression constitute two problems that can be solved by the training of probabilistic models on a first dataset with subsequent likelihood evaluation on a second dataset, where data distributions differ. By defining the generalization of probabilistic models in terms of likelihood we show that, in the case of image models, the OOD generalization ability is dominated by local features. This motivates our proposal of a Local Autoregressive model that exclusively models local image features towards improving OOD performance. We apply the proposed model to OOD detection tasks and achieve state-of-the-art unsupervised OOD detection performance without the introduction of additional data. Additionally, we employ our model to build a new lossless image compressor: NeLLoC (Neural Local Lossless Compressor) and report state-of-the-art compression rates and model size.
References in corpus (19)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
- Generating Long Sequences with Sparse Transformers
- Deep Anomaly Detection with Outlier Exposure
- WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
- Input complexity and out-of-distribution detection with likelihood-based generative models
- Asymmetric numeral systems
- Uncertainty in the Variational Information Bottleneck
- Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variables
- Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
- Practical Lossless Compression with Latent Variables using Bits Back Coding
- IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
- Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Features
- Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices
- Understanding the (un)interpretability of natural image distributions using generative models
- HiLLoC: Lossless Image Compression with Hierarchical Latent Variable Models
- Multiscale Score Matching for Out-of-Distribution Detection
- Reducing the Computational Cost of Deep Generative Models with Binary Neural Networks