Stochastic Conditional Generative Networks with Basis Decomposition

Abstract

While generative adversarial networks (GANs) have revolutionized machine learning, a number of open questions remain to fully understand them and exploit their power. One of these questions is how to efficiently achieve proper diversity and sampling of the multi-mode data space. To address this, we introduce BasisGAN, a stochastic conditional multi-mode image generator. By exploiting the observation that a convolutional filter can be well approximated as a linear combination of a small set of basis elements, we learn a plug-and-played basis generator to stochastically generate basis elements, with just a few hundred of parameters, to fully embed stochasticity into convolutional filters. By sampling basis elements instead of filters, we dramatically reduce the cost of modeling the parameter space with no sacrifice on either image diversity or fidelity. To illustrate this proposed plug-and-play framework, we construct variants of BasisGAN based on state-of-the-art conditional image generation networks, and train the networks by simply plugging in a basis generator, without additional auxiliary components, hyperparameters, or training objectives. The experimental success is complemented with theoretical results indicating how the perturbations introduced by the proposed sampling of basis elements can propagate to the appearance of generated images.

Published as a conference paper at ICLR 2020

References in corpus (10)

• Conditional Generative Adversarial Nets
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• Toward Multimodal Image-to-Image Translation
• Diversity-Sensitive Conditional Generative Adversarial Networks
• Global Optimality in Tensor Factorization, Deep Learning, and Beyond
• Mathematics of Deep Learning
• DCFNet: Deep Neural Network with Decomposed Convolutional Filters
• TAFE-Net: Task-Aware Feature Embeddings for Low Shot Learning

Cited by in corpus (3)

• Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
• Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpoint
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data