Semi-Supervised Learning with Normalizing Flows

Abstract

Normalizing flows transform a latent distribution through an invertible neural network for a flexible and pleasingly simple approach to generative modelling, while preserving an exact likelihood. We propose FlowGMM, an end-to-end approach to generative semi supervised learning with normalizing flows, using a latent Gaussian mixture model. FlowGMM is distinct in its simplicity, unified treatment of labelled and unlabelled data with an exact likelihood, interpretability, and broad applicability beyond image data. We show promising results on a wide range of applications, including AG-News and Yahoo Answers text data, tabular data, and semi-supervised image classification. We also show that FlowGMM can discover interpretable structure, provide real-time optimization-free feature visualizations, and specify well calibrated predictive distributions.

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• On Calibration of Modern Neural Networks
• Unsupervised Data Augmentation for Consistency Training
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Semi-Supervised Learning with Deep Generative Models
• Temporal Ensembling for Semi-Supervised Learning
• NICE: Non-linear Independent Components Estimation
• Interpolation Consistency Training for Semi-Supervised Learning
• Density estimation using Real NVP
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• Invertible Residual Networks
• There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average
• Semi-Conditional Normalizing Flows for Semi-Supervised Learning

Cited by in corpus (6)

• TabTransformer: Tabular Data Modeling Using Contextual Embeddings
• On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition
• An Improved Semi-Supervised VAE for Learning Disentangled Representations
• Semi-supervised Learning by Latent Space Energy-Based Model of Symbol-Vector Coupling
• Variational Mixture of Normalizing Flows
• Resampling Base Distributions of Normalizing Flows