Glow: Generative Flow with Invertible 1x1 Convolutions

Abstract

Flow-based generative models (Dinh et al., 2014) are conceptually attractive due to tractability of the exact log-likelihood, tractability of exact latent-variable inference, and parallelizability of both training and synthesis. In this paper we propose Glow, a simple type of generative flow using an invertible 1x1 convolution. Using our method we demonstrate a significant improvement in log-likelihood on standard benchmarks. Perhaps most strikingly, we demonstrate that a generative model optimized towards the plain log-likelihood objective is capable of efficient realistic-looking synthesis and manipulation of large images. The code for our model is available at https://github.com/openai/glow

arXiv comment15 pages; fixed typo in abstract

References in corpus (10)

• Auto-Encoding Variational Bayes
• WaveNet: A Generative Model for Raw Audio
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• NICE: Non-linear Independent Components Estimation
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Variational Inference with Normalizing Flows
• Density estimation using Real NVP
• The Reversible Residual Network: Backpropagation Without Storing Activations
• Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models

Cited by in corpus (24)

• The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
• A Tutorial on Deep Latent Variable Models of Natural Language
• On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition
• Self-Supervised Variational Auto-Encoders
• A Case for the Score: Identifying Image Anomalies using Variational Autoencoder Gradients
• Learning Symmetries of Classical Integrable Systems
• Robotic Table Tennis: A Case Study into a High Speed Learning System
• Coarse-to-Fine Proposal Refinement Framework for Audio Temporal Forgery Detection and Localization
• Training Invertible Neural Networks as Autoencoders
• A Survey on Deep Domain Adaptation and Tiny Object Detection Challenges, Techniques and Datasets
• ChartPointFlow for Topology-Aware 3D Point Cloud Generation
• Recapture as You Want
• Failure Modes of Variational Autoencoders and Their Effects on Downstream Tasks
• Improving Sequential Latent Variable Models with Autoregressive Flows
• Deep Generative Models Strike Back! Improving Understanding and Evaluation in Light of Unmet Expectations for OoD Data
• TreeFlow: Going beyond Tree-based Gaussian Probabilistic Regression
• On Out-of-distribution Detection with Energy-based Models
• Inference and De-Noising of Non-Gaussian Particle Distribution Functions: A Generative Modeling Approach
• Exploring Adversarial Examples via Invertible Neural Networks
• TzK Flow - Conditional Generative Model
• Out-of-Distribution Detection of Melanoma using Normalizing Flows
• GANtron: Emotional Speech Synthesis with Generative Adversarial Networks
• Synthesizing Photorealistic Images with Deep Generative Learning
• Invertible Surrogate Models: Joint surrogate modelling and reconstruction of Laser-Wakefield Acceleration by invertible neural networks