MintNet: Building Invertible Neural Networks with Masked Convolutions
arXiv:1907.07945
Abstract
We propose a new way of constructing invertible neural networks by combining simple building blocks with a novel set of composition rules. This leads to a rich set of invertible architectures, including those similar to ResNets. Inversion is achieved with a locally convergent iterative procedure that is parallelizable and very fast in practice. Additionally, the determinant of the Jacobian can be computed analytically and efficiently, enabling their generative use as flow models. To demonstrate their flexibility, we show that our invertible neural networks are competitive with ResNets on MNIST and CIFAR-10 classification. When trained as generative models, our invertible networks achieve competitive likelihoods on MNIST, CIFAR-10 and ImageNet 32x32, with bits per dimension of 0.98, 3.32 and 4.06 respectively.
arXiv commentNeurIPS 2019
References in corpus (17)
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- SGDR: Stochastic Gradient Descent with Warm Restarts
- On the Convergence of Adam and Beyond
- Pixel Recurrent Neural Networks
- NICE: Non-linear Independent Components Estimation
- Variational Inference with Normalizing Flows
- Density estimation using Real NVP
- The Reversible Residual Network: Backpropagation Without Storing Activations
- Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
- Invertible Residual Networks
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- Sylvester Normalizing Flows for Variational Inference
- On the Convergence Proof of AMSGrad and a New Version
- A-NICE-MC: Adversarial Training for MCMC
- Emerging Convolutions for Generative Normalizing Flows
- Reversible Recurrent Neural Networks
- MaCow: Masked Convolutional Generative Flow