Bigeminal Priors Variational auto-encoder
arXiv:2010.01819
Abstract
Variational auto-encoders (VAEs) are an influential and generally-used class of likelihood-based generative models in unsupervised learning. The likelihood-based generative models have been reported to be highly robust to the out-of-distribution (OOD) inputs and can be a detector by assuming that the model assigns higher likelihoods to the samples from the in-distribution (ID) dataset than an OOD dataset. However, recent works reported a phenomenon that VAE recognizes some OOD samples as ID by assigning a higher likelihood to the OOD inputs compared to the one from ID. In this work, we introduce a new model, namely Bigeminal Priors Variational auto-encoder (BPVAE), to address this phenomenon. The BPVAE aims to enhance the robustness of the VAEs by combing the power of VAE with the two independent priors that belong to the training dataset and simple dataset, which complexity is lower than the training dataset, respectively. BPVAE learns two datasets'features, assigning a higher likelihood for the training dataset than the simple dataset. In this way, we can use BPVAE's density estimate for detecting the OOD samples. Quantitative experimental results suggest that our model has better generalization capability and stronger robustness than the standard VAEs, proving the effectiveness of the proposed approach of hybrid learning by collaborative priors. Overall, this work paves a new avenue to potentially overcome the OOD problem via multiple latent priors modeling.
References in corpus (14)
- Auto-Encoding Variational Bayes
- Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory
- Conditional Image Generation with PixelCNN Decoders
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
- Deep Anomaly Detection with Outlier Exposure
- WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
- BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling
- Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
- Noise Contrastive Priors for Functional Uncertainty
- Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
- Unsupervised Out-of-Distribution Detection with Batch Normalization
- Resampled Priors for Variational Autoencoders
- Out-of-Distribution Detection Using Neural Rendering Generative Models