Conditional Generative Modeling via Learning the Latent Space

Abstract

Although deep learning has achieved appealing results on several machine learning tasks, most of the models are deterministic at inference, limiting their application to single-modal settings. We propose a novel general-purpose framework for conditional generation in multimodal spaces, that uses latent variables to model generalizable learning patterns while minimizing a family of regression cost functions. At inference, the latent variables are optimized to find optimal solutions corresponding to multiple output modes. Compared to existing generative solutions, in multimodal spaces, our approach demonstrates faster and stable convergence, and can learn better representations for downstream tasks. Importantly, it provides a simple generic model that can beat highly engineered pipelines tailored using domain expertise on a variety of tasks, while generating diverse outputs. Our codes will be released.

References in corpus (17)

• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Toward Multimodal Image-to-Image Translation
• Image Inpainting via Generative Multi-column Convolutional Neural Networks
• On Convergence and Stability of GANs
• DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications
• Do GANs actually learn the distribution? An empirical study
• BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling
• PixelNet: Representation of the pixels, by the pixels, and for the pixels
• Improving Generalization and Stability of Generative Adversarial Networks
• Convergence Problems with Generative Adversarial Networks (GANs)
• Smoothness and Stability in GANs
• Understanding the Limitations of Conditional Generative Models
• Harmonizing Maximum Likelihood with GANs for Multimodal Conditional Generation
• Spectral-GANs for High-Resolution 3D Point-cloud Generation
• Domain Embedded Multi-model Generative Adversarial Networks for Image-based Face Inpainting
• Volumetric Convolution: Automatic Representation Learning in Unit Ball