Hybrid Models with Deep and Invertible Features

Abstract

We propose a neural hybrid model consisting of a linear model defined on a set of features computed by a deep, invertible transformation (i.e. a normalizing flow). An attractive property of our model is that both p(features), the density of the features, and p(targets | features), the predictive distribution, can be computed exactly in a single feed-forward pass. We show that our hybrid model, despite the invertibility constraints, achieves similar accuracy to purely predictive models. Moreover the generative component remains a good model of the input features despite the hybrid optimization objective. This offers additional capabilities such as detection of out-of-distribution inputs and enabling semi-supervised learning. The availability of the exact joint density p(targets, features) also allows us to compute many quantities readily, making our hybrid model a useful building block for downstream applications of probabilistic deep learning.

ICML 2019

Cited by in corpus (21)

• Normalizing Flows for Probabilistic Modeling and Inference
• Invertible Residual Networks
• Contrastive Training for Improved Out-of-Distribution Detection
• Data-driven Cloud Clustering via a Rotationally Invariant Autoencoder
• Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models
• VFlow: More Expressive Generative Flows with Variational Data Augmentation
• Understanding the Limitations of Conditional Generative Models
• Densely connected normalizing flows
• Invertible DenseNets with Concatenated LipSwish
• Further Analysis of Outlier Detection with Deep Generative Models
• InfoCNF: An Efficient Conditional Continuous Normalizing Flow with Adaptive Solvers
• Variational Inference for Computational Imaging Inverse Problems
• Training Normalizing Flows with the Information Bottleneck for Competitive Generative Classification
• Task-agnostic Continual Learning with Hybrid Probabilistic Models
• Invertible DenseNets
• GAN-based Priors for Quantifying Uncertainty
• Types of Out-of-Distribution Texts and How to Detect Them
• Revisiting Explicit Regularization in Neural Networks for Well-Calibrated Predictive Uncertainty
• Ordering Dimensions with Nested Dropout Normalizing Flows
• Learning the Prediction Distribution for Semi-Supervised Learning with Normalising Flows
• Semi-Supervised Few-Shot Classification with Deep Invertible Hybrid Models