Unsupervised Discovery of Interpretable Directions in the GAN Latent Space

Abstract

The latent spaces of GAN models often have semantically meaningful directions. Moving in these directions corresponds to human-interpretable image transformations, such as zooming or recoloring, enabling a more controllable generation process. However, the discovery of such directions is currently performed in a supervised manner, requiring human labels, pretrained models, or some form of self-supervision. These requirements severely restrict a range of directions existing approaches can discover. In this paper, we introduce an unsupervised method to identify interpretable directions in the latent space of a pretrained GAN model. By a simple model-agnostic procedure, we find directions corresponding to sensible semantic manipulations without any form of (self-)supervision. Furthermore, we reveal several non-trivial findings, which would be difficult to obtain by existing methods, e.g., a direction corresponding to background removal. As an immediate practical benefit of our work, we show how to exploit this finding to achieve competitive performance for weakly-supervised saliency detection.

References in corpus (10)

• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Unsupervised Discovery of Interpretable Directions in the GAN Latent Space
• GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
• Video-to-Video Synthesis
• Controlling generative models with continuous factors of variations
• Interpreting the Latent Space of GANs for Semantic Face Editing
• Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
• A Spectral Regularizer for Unsupervised Disentanglement
• RPGAN: GANs Interpretability via Random Routing

Cited by in corpus (29)

• GANSpace: Discovering Interpretable GAN Controls
• Unsupervised Discovery of Interpretable Directions in the GAN Latent Space
• A Good Image Generator Is What You Need for High-Resolution Video Synthesis
• Closed-Form Factorization of Latent Semantics in GANs
• A Survey on Neural Network Interpretability
• CA-GAN: Weakly Supervised Color Aware GAN for Controllable Makeup Transfer
• GANzilla: User-Driven Direction Discovery in Generative Adversarial Networks
• High-Fidelity GAN Inversion for Image Attribute Editing
• InMoDeGAN: Interpretable Motion Decomposition Generative Adversarial Network for Video Generation
• Only a Matter of Style: Age Transformation Using a Style-Based Regression Model
• GANravel: User-Driven Direction Disentanglement in Generative Adversarial Networks
• Attribute-specific Control Units in StyleGAN for Fine-grained Image Manipulation
• Knowledge-integrated AutoEncoder Model
• Compositional Transformers for Scene Generation
• Feature Unlearning for Pre-trained GANs and VAEs
• MotionVideoGAN: A Novel Video Generator Based on the Motion Space Learned from Image Pairs
• Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs
• Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs
• Domain-Scalable Unpaired Image Translation via Latent Space Anchoring
• Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation
• Generating Gameplay-Relevant Art Assets with Transfer Learning
• Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Model
• Controlling the Output of a Generative Model by Latent Feature Vector Shifting
• Ensembling with Deep Generative Views
• Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing
• Low-Rank Subspaces in GANs
• SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing
• Bag-of-Vectors Autoencoders for Unsupervised Conditional Text Generation
• Manipulating Vehicle 3D Shapes through Latent Space Editing