Autoregressive Quantile Networks for Generative Modeling

Abstract

We introduce autoregressive implicit quantile networks (AIQN), a fundamentally different approach to generative modeling than those commonly used, that implicitly captures the distribution using quantile regression. AIQN is able to achieve superior perceptual quality and improvements in evaluation metrics, without incurring a loss of sample diversity. The method can be applied to many existing models and architectures. In this work we extend the PixelCNN model with AIQN and demonstrate results on CIFAR-10 and ImageNet using Inception score, FID, non-cherry-picked samples, and inpainting results. We consistently observe that AIQN yields a highly stable algorithm that improves perceptual quality while maintaining a highly diverse distribution.

arXiv commentICML 2018

References in corpus (20)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• WaveNet: A Generative Model for Raw Audio
• Neural Discrete Representation Learning
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Improved Techniques for Training GANs
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Adversarially Learned Inference
• A note on the evaluation of generative models
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Implicit Quantile Networks for Distributional Reinforcement Learning
• Towards Deeper Understanding of Variational Autoencoding Models
• From optimal transport to generative modeling: the VEGAN cookbook
• PixelSNAIL: An Improved Autoregressive Generative Model
• Improving GANs Using Optimal Transport
• GAN and VAE from an Optimal Transport Point of View

Cited by in corpus (14)

• Classification Accuracy Score for Conditional Generative Models
• Implicit Generation and Generalization in Energy-Based Models
• Improved Contrastive Divergence Training of Energy Based Models
• Generative Adversarial Networks for Financial Trading Strategies Fine-Tuning and Combination
• Deep Generative Quantile-Copula Models for Probabilistic Forecasting
• Probabilistic Time Series Forecasting with Implicit Quantile Networks
• Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling by Exploring Energy of the Discriminator
• Super-resolution Variational Auto-Encoders
• VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
• Generalized Energy Based Models
• Improved Autoregressive Modeling with Distribution Smoothing
• Adapting deep generative approaches for getting synthetic data with realistic marginal distributions
• Uncertainty Intervals for Graph-based Spatio-Temporal Traffic Prediction
• Learning Policies through Quantile Regression