General Invertible Transformations for Flow-based Generative Modeling

Abstract

In this paper, we present a new class of invertible transformations with an application to flow-based generative models. We indicate that many well-known invertible transformations in reversible logic and reversible neural networks could be derived from our proposition. Next, we propose two new coupling layers that are important building blocks of flow-based generative models. In the experiments on digit data, we present how these new coupling layers could be used in Integer Discrete Flows (IDF), and that they achieve better results than standard coupling layers used in IDF and RealNVP.

Code: https://github.com/jmtomczak/git_flow, accepted to INNF+ 2021 at ICML

References in corpus (10)

• Conditional Image Generation with PixelCNN Decoders
• Density estimation using Real NVP
• The Reversible Residual Network: Backpropagation Without Storing Activations
• Residual Flows for Invertible Generative Modeling
• Sylvester Normalizing Flows for Variational Inference
• Improving Variational Auto-Encoders using Householder Flow
• IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
• Reversible Recurrent Neural Networks
• Improving Variational Auto-Encoders using convex combination linear Inverse Autoregressive Flow
• Invertible DenseNets with Concatenated LipSwish