Closed-Form Factorization of Latent Semantics in GANs

Abstract

A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In order to identify such latent dimensions for image editing, previous methods typically annotate a collection of synthesized samples and train linear classifiers in the latent space. However, they require a clear definition of the target attribute as well as the corresponding manual annotations, limiting their applications in practice. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. In particular, we take a closer look into the generation mechanism of GANs and further propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights. With a lightning-fast implementation, our approach is capable of not only finding semantically meaningful dimensions comparably to the state-of-the-art supervised methods, but also resulting in far more versatile concepts across multiple GAN models trained on a wide range of datasets.

CVPR 2021 camera-ready

References in corpus (5)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Unsupervised Discovery of Interpretable Directions in the GAN Latent Space
• GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
• Controlling generative models with continuous factors of variations

Cited by in corpus (17)

• Learning Disentangled Representations in the Imaging Domain
• CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis
• A Good Image Generator Is What You Need for High-Resolution Video Synthesis
• GANzilla: User-Driven Direction Discovery in Generative Adversarial Networks
• Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models
• CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
• GANravel: User-Driven Direction Disentanglement in Generative Adversarial Networks
• Explaining in Style: Training a GAN to explain a classifier in StyleSpace
• Domain-Scalable Unpaired Image Translation via Latent Space Anchoring
• Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation
• ShapeEditer: a StyleGAN Encoder for Face Swapping
• DIFAI: Diverse Facial Inpainting using StyleGAN Inversion
• Auditing AI models for Verified Deployment under Semantic Specifications
• Disentangled Face Attribute Editing via Instance-Aware Latent Space Search
• Ensembling with Deep Generative Views
• Lifting 2D StyleGAN for 3D-Aware Face Generation
• Linear Semantics in Generative Adversarial Networks