Semi-Supervised StyleGAN for Disentanglement Learning

Abstract

Disentanglement learning is crucial for obtaining disentangled representations and controllable generation. Current disentanglement methods face several inherent limitations: difficulty with high-resolution images, primarily focusing on learning disentangled representations, and non-identifiability due to the unsupervised setting. To alleviate these limitations, we design new architectures and loss functions based on StyleGAN (Karras et al., 2019), for semi-supervised high-resolution disentanglement learning. We create two complex high-resolution synthetic datasets for systematic testing. We investigate the impact of limited supervision and find that using only 0.25%~2.5% of labeled data is sufficient for good disentanglement on both synthetic and real datasets. We propose new metrics to quantify generator controllability, and observe there may exist a crucial trade-off between disentangled representation learning and controllable generation. We also consider semantic fine-grained image editing to achieve better generalization to unseen images.

arXiv commentICML 2020, 21 pages. Project page: https://sites.google.com/nvidia.com/semi-stylegan

References in corpus (12)

• Conditional Generative Adversarial Nets
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Semi-Supervised Learning with Deep Generative Models
• Deep Convolutional Inverse Graphics Network
• Disentangling by Factorising
• FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age
• Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
• Learning Deep Disentangled Embeddings with the F-Statistic Loss
• DNA-GAN: Learning Disentangled Representations from Multi-Attribute Images
• On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset

Cited by in corpus (1)

• Disentangling representations of retinal images with generative models