GANSpace: Discovering Interpretable GAN Controls

Abstract

This paper describes a simple technique to analyze Generative Adversarial Networks (GANs) and create interpretable controls for image synthesis, such as change of viewpoint, aging, lighting, and time of day. We identify important latent directions based on Principal Components Analysis (PCA) applied either in latent space or feature space. Then, we show that a large number of interpretable controls can be defined by layer-wise perturbation along the principal directions. Moreover, we show that BigGAN can be controlled with layer-wise inputs in a StyleGAN-like manner. We show results on different GANs trained on various datasets, and demonstrate good qualitative matches to edit directions found through earlier supervised approaches.

Accepted to NeurIPS 2020

References in corpus (4)

• Deep Convolutional Inverse Graphics Network
• Unsupervised Discovery of Interpretable Directions in the GAN Latent Space
• Controlling generative models with continuous factors of variations
• Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space

Cited by in corpus (42)

• Fixing Data Augmentation to Improve Adversarial Robustness
• A Good Image Generator Is What You Need for High-Resolution Video Synthesis
• E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion
• Image-to-Image Translation with Disentangled Latent Vectors for Face Editing
• A Survey on Neural Network Interpretability
• Talking Head from Speech Audio using a Pre-trained Image Generator
• Guided Co-Modulated GAN for 360° Field of View Extrapolation
• Controllable Data Generation by Deep Learning: A Review
• GANzilla: User-Driven Direction Discovery in Generative Adversarial Networks
• TexSliders: Diffusion-Based Texture Editing in CLIP Space
• Nine Potential Pitfalls when Designing Human-AI Co-Creative Systems
• Adversarial Defense by Latent Style Transformations
• GANravel: User-Driven Direction Disentanglement in Generative Adversarial Networks
• CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
• Attribute-specific Control Units in StyleGAN for Fine-grained Image Manipulation
• Gaussian Splatting Decoder for 3D-aware Generative Adversarial Networks
• Learning and controlling the source-filter representation of speech with a variational autoencoder
• Feature Unlearning for Pre-trained GANs and VAEs
• Stylized Face Sketch Extraction via Generative Prior with Limited Data
• Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks
• MammoGANesis: Controlled Generation of High-Resolution Mammograms for Radiology Education
• FACEMUG: A Multimodal Generative and Fusion Framework for Local Facial Editing
• 3D Generative Model Latent Disentanglement via Local Eigenprojection
• Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation
• Improving Robustness using Generated Data
• Example-Based Framework for Perceptually Guided Audio Texture Generation
• ShapeEditer: a StyleGAN Encoder for Face Swapping
• DIFAI: Diverse Facial Inpainting using StyleGAN Inversion
• Continuous Facial Motion Deblurring
• Solving inverse problems with deep neural networks driven by sparse signal decomposition in a physics-based dictionary
• Concept Lens: Visually Analyzing the Consistency of Semantic Manipulation in GANs
• Robust Training Using Natural Transformation
• An Assessment of GANs for Identity-related Applications
• Neuro-Symbolic Generative Art: A Preliminary Study
• Learning Gaze-aware Compositional GAN
• One Shot Face Swapping on Megapixels
• SAT3D: Image-driven Semantic Attribute Transfer in 3D
• A Reference-Based 3D Semantic-Aware Framework for Accurate Local Facial Attribute Editing
• StylePart: Image-based Shape Part Manipulation
• Towards Disentangling Latent Space for Unsupervised Semantic Face Editing
• FaceCook: Face Generation Based on Linear Scaling Factors
• StyleDiff: Attribute Comparison Between Unlabeled Datasets in Latent Disentangled Space