Stein Latent Optimization for Generative Adversarial Networks

Abstract

Generative adversarial networks (GANs) with clustered latent spaces can perform conditional generation in a completely unsupervised manner. In the real world, the salient attributes of unlabeled data can be imbalanced. However, most of existing unsupervised conditional GANs cannot cluster attributes of these data in their latent spaces properly because they assume uniform distributions of the attributes. To address this problem, we theoretically derive Stein latent optimization that provides reparameterizable gradient estimations of the latent distribution parameters assuming a Gaussian mixture prior in a continuous latent space. Structurally, we introduce an encoder network and novel unsupervised conditional contrastive loss to ensure that data generated from a single mixture component represent a single attribute. We confirm that the proposed method, named Stein Latent Optimization for GANs (SLOGAN), successfully learns balanced or imbalanced attributes and achieves state-of-the-art unsupervised conditional generation performance even in the absence of attribute information (e.g., the imbalance ratio). Moreover, we demonstrate that the attributes to be learned can be manipulated using a small amount of probe data.

arXiv commentICLR 2022 camera ready

References in corpus (18)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Categorical Reparameterization with Gumbel-Softmax
• Conditional Image Synthesis With Auxiliary Classifier GANs
• On Variational Bounds of Mutual Information
• On the regularization of Wasserstein GANs
• Implicit Reparameterization Gradients
• ContraGAN: Contrastive Learning for Conditional Image Generation
• Deep Robust Clustering by Contrastive Learning
• Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations
• Smoothed Geometry for Robust Attribution
• Adversarial Lipschitz Regularization
• Hierarchical Adversarially Learned Inference
• HexaGAN: Generative Adversarial Nets for Real World Classification
• Effect of The Latent Structure on Clustering with GANs
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data
• Stein's Lemma for the Reparameterization Trick with Exponential Family Mixtures
• CoDeGAN: Contrastive Disentanglement for Generative Adversarial Network