MaCow: Masked Convolutional Generative Flow
arXiv:1902.04208
Abstract
Flow-based generative models, conceptually attractive due to tractability of both the exact log-likelihood computation and latent-variable inference, and efficiency of both training and sampling, has led to a number of impressive empirical successes and spawned many advanced variants and theoretical investigations. Despite their computational efficiency, the density estimation performance of flow-based generative models significantly falls behind those of state-of-the-art autoregressive models. In this work, we introduce masked convolutional generative flow (MaCow), a simple yet effective architecture of generative flow using masked convolution. By restricting the local connectivity in a small kernel, MaCow enjoys the properties of fast and stable training, and efficient sampling, while achieving significant improvements over Glow for density estimation on standard image benchmarks, considerably narrowing the gap to autoregressive models.
arXiv commentIn Proceedings of Thirty-third Conference on Neural Information Processing Systems (NeurIPS-2019)
References in corpus (19)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Auto-Encoding Variational Bayes
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- WaveNet: A Generative Model for Raw Audio
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- Conditional Image Generation with PixelCNN Decoders
- Pixel Recurrent Neural Networks
- NICE: Non-linear Independent Components Estimation
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- Density estimation using Real NVP
- MADE: Masked Autoencoder for Distribution Estimation
- RNADE: The real-valued neural autoregressive density-estimator
- Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling
- PixelSNAIL: An Improved Autoregressive Generative Model
- Latent Normalizing Flows for Discrete Sequences
- Emerging Convolutions for Generative Normalizing Flows
- MAE: Mutual Posterior-Divergence Regularization for Variational AutoEncoders