InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Abstract

This paper describes InfoGAN, an information-theoretic extension to the Generative Adversarial Network that is able to learn disentangled representations in a completely unsupervised manner. InfoGAN is a generative adversarial network that also maximizes the mutual information between a small subset of the latent variables and the observation. We derive a lower bound to the mutual information objective that can be optimized efficiently, and show that our training procedure can be interpreted as a variation of the Wake-Sleep algorithm. Specifically, InfoGAN successfully disentangles writing styles from digit shapes on the MNIST dataset, pose from lighting of 3D rendered images, and background digits from the central digit on the SVHN dataset. It also discovers visual concepts that include hair styles, presence/absence of eyeglasses, and emotions on the CelebA face dataset. Experiments show that InfoGAN learns interpretable representations that are competitive with representations learned by existing fully supervised methods.

References in corpus (1)

• Adam: A Method for Stochastic Optimization

Cited by in corpus (276)

• Generative Adversarial Networks: An Overview
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Toward Multimodal Image-to-Image Translation
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• Disentangling by Factorising
• Deep Reinforcement Learning: An Overview
• Unsupervised Image-to-Image Translation Networks
• Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
• Deep Generative Design: Integration of Topology Optimization and Generative Models
• Understanding the Role of Individual Units in a Deep Neural Network
• Recent Advances in Autoencoder-Based Representation Learning
• InfoVAE: Information Maximizing Variational Autoencoders
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• State Representation Learning for Control: An Overview
• MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation
• Multimodal Unsupervised Image-to-Image Translation
• Concept Whitening for Interpretable Image Recognition
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Microstructural Materials Design via Deep Adversarial Learning Methodology
• MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification
• Multi-Object Representation Learning with Iterative Variational Inference
• Triple Generative Adversarial Nets
• Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro
• Disentangled Representation Learning in Cardiac Image Analysis
• DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
• TrajGAIL: Generating Urban Vehicle Trajectories using Generative Adversarial Imitation Learning
• DisenHAN: Disentangled Heterogeneous Graph Attention Network for Recommendation
• An Introduction to Image Synthesis with Generative Adversarial Nets
• Least Squares Generative Adversarial Networks
• Third-Person Imitation Learning
• Toward Controlled Generation of Text
• Towards Deep Symbolic Reinforcement Learning
• Deep Identity-aware Transfer of Facial Attributes
• Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities
• InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
• On the regularization of Wasserstein GANs
• Generative Adversarial Networks (GANs) in Networking: A Comprehensive Survey & Evaluation
• A Bayesian Data Augmentation Approach for Learning Deep Models
• Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks
• A survey of synthetic data augmentation methods in computer vision
• BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images
• Deep Semi Supervised Generative Learning for Automated PD-L1 Tumor Cell Scoring on NSCLC Tissue Needle Biopsies
• Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning
• Age Progression/Regression by Conditional Adversarial Autoencoder
• Generative Models for Automatic Chemical Design
• Deep Variational Canonical Correlation Analysis
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• Probabilistic Forecasting of Sensory Data with Generative Adversarial Networks - ForGAN
• Connecting Generative Adversarial Networks and Actor-Critic Methods
• Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
• GP-GAN: Towards Realistic High-Resolution Image Blending
• Image-to-image translation for cross-domain disentanglement
• Learning Plannable Representations with Causal InfoGAN
• Learning Predictive Representations for Deformable Objects Using Contrastive Estimation
• Controlling generative models with continuous factors of variations
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
• Deep learning for presumed probability density function models
• Curiosity Driven Exploration of Learned Disentangled Goal Spaces
• DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
• Style Aggregated Network for Facial Landmark Detection
• Pseudo-healthy synthesis with pathology disentanglement and adversarial learning
• Restructuring Batch Normalization to Accelerate CNN Training
• Neural Expectation Maximization
• Evaluation Metrics for Conditional Image Generation
• IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks
• Learning Beyond Human Expertise with Generative Models for Dental Restorations
• Unsupervised Text Style Transfer using Language Models as Discriminators
• Stacked Generative Adversarial Networks
• Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning
• Towards Large-Pose Face Frontalization in the Wild
• Multi-View Image Generation from a Single-View
• Ensembles of Generative Adversarial Networks
• Adversarial Symmetric Variational Autoencoder
• Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives
• Learning Hierarchical Features from Generative Models
• A Comprehensive Survey of Machine Learning Applied to Radar Signal Processing
• Scene Text Synthesis for Efficient and Effective Deep Network Training
• Towards Open-Set Identity Preserving Face Synthesis
• Conditional Generation of Medical Images via Disentangled Adversarial Inference
• Autoregressive Quantile Networks for Generative Modeling
• The Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Models
• Auto-Encoding Total Correlation Explanation
• CA-GAN: Weakly Supervised Color Aware GAN for Controllable Makeup Transfer
• Unsupervised Visual Attribute Transfer with Reconfigurable Generative Adversarial Networks
• Probabilistic Autoencoder
• Conditional Adversarial Synthesis of 3D Facial Action Units
• Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusion
• Amortized Variational Inference: A Systematic Review
• Pose-Normalized Image Generation for Person Re-identification
• Improving Lesion Segmentation for Diabetic Retinopathy using Adversarial Learning
• Improved Training of Generative Adversarial Networks Using Representative Features
• 3D Object Reconstruction from a Single Depth View with Adversarial Learning
• Generative machine learning methods for multivariate ensemble post-processing
• Unsupervised Portrait Shadow Removal via Generative Priors
• Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial Networks
• McGan: Mean and Covariance Feature Matching GAN
• Comparative Study on Generative Adversarial Networks
• Speech-Driven Facial Reenactment Using Conditional Generative Adversarial Networks
• FCC-GAN: A Fully Connected and Convolutional Net Architecture for GANs
• Inverting The Generator Of A Generative Adversarial Network (II)
• Learning Texture Manifolds with the Periodic Spatial GAN
• Generative Adversarial Networks: A Survey Towards Private and Secure Applications
• DiscrimNet: Semi-Supervised Action Recognition from Videos using Generative Adversarial Networks
• Relevance Factor VAE: Learning and Identifying Disentangled Factors
• Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking
• Learning Not to Learn: Training Deep Neural Networks with Biased Data
• An Adversarial Neuro-Tensorial Approach For Learning Disentangled Representations
• Transferring GANs: generating images from limited data
• Investigating Cosmological GAN Emulators Using Latent Space Interpolation
• Knowledge Consistency between Neural Networks and Beyond
• Learning Disentangled Representations with Reference-Based Variational Autoencoders
• ProstateGAN: Mitigating Data Bias via Prostate Diffusion Imaging Synthesis with Generative Adversarial Networks
• eCommerceGAN : A Generative Adversarial Network for E-commerce
• CariGAN: Caricature Generation through Weakly Paired Adversarial Learning
• Unsupervised Learning of Neural Networks to Explain Neural Networks
• Closed-Loop Data Transcription to an LDR via Minimaxing Rate Reduction
• Style Generator Inversion for Image Enhancement and Animation
• Message Passing Multi-Agent GANs
• O-GAN: Extremely Concise Approach for Auto-Encoding Generative Adversarial Networks
• Disentangled Recurrent Wasserstein Autoencoder
• On the Effectiveness of Least Squares Generative Adversarial Networks
• Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
• Ranking CGANs: Subjective Control over Semantic Image Attributes
• VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention
• Local and non-local dependency learning and emergence of rule-like representations in speech data by Deep Convolutional Generative Adversarial Networks
• Associative Compression Networks for Representation Learning
• Solving Linear Inverse Problems Using GAN Priors: An Algorithm with Provable Guarantees
• Explicit Disentanglement of Appearance and Perspective in Generative Models
• Deep Learning of Dynamic Subsurface Flow via Theory-guided Generative Adversarial Network
• DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data
• Disentangling Factors of Variation by Mixing Them
• Semantics Disentangling for Text-to-Image Generation
• Hyperparameter Auto-tuning in Self-Supervised Robotic Learning
• Learning Part Generation and Assembly for Structure-aware Shape Synthesis
• Ways of Conditioning Generative Adversarial Networks
• Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling
• Learning and controlling the source-filter representation of speech with a variational autoencoder
• Learning Residual Images for Face Attribute Manipulation
• Learning to Write Stylized Chinese Characters by Reading a Handful of Examples
• Controllable Generative Adversarial Network
• Human Action Generation with Generative Adversarial Networks
• Bounded Information Rate Variational Autoencoders
• Latent Space Factorisation and Manipulation via Matrix Subspace Projection
• One-Class Adversarial Nets for Fraud Detection
• MISO: Mutual Information Loss with Stochastic Style Representations for Multimodal Image-to-Image Translation
• DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks (with Supplementary Materials)
• TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation
• Is Discriminator a Good Feature Extractor?
• Multi-Agent Reinforcement Learning with Multi-Step Generative Models
• VAE-Info-cGAN: Generating Synthetic Images by Combining Pixel-level and Feature-level Geospatial Conditional Inputs
• Imparting Interpretability to Word Embeddings while Preserving Semantic Structure
• Disentangling and Learning Robust Representations with Natural Clustering
• Deep Image Clustering with Category-Style Representation
• GDPP: Learning Diverse Generations Using Determinantal Point Process
• Image Generation and Editing with Variational Info Generative AdversarialNetworks
• Learning from Multi-domain Artistic Images for Arbitrary Style Transfer
• Faces à la Carte: Text-to-Face Generation via Attribute Disentanglement
• Investigation of Using Disentangled and Interpretable Representations for One-shot Cross-lingual Voice Conversion
• Towards a Unified Evaluation of Explanation Methods without Ground Truth
• Learning Compositional Visual Concepts with Mutual Consistency
• Towards Interpretable Face Recognition
• Pose-Guided High-Resolution Appearance Transfer via Progressive Training
• Commutative Lie Group VAE for Disentanglement Learning
• Privacy-Preserving Representation Learning on Graphs: A Mutual Information Perspective
• SDIT: Scalable and Diverse Cross-domain Image Translation
• Generative adversarial networks for generation and classification of physical rehabilitation movement episodes
• PixelNN: Example-based Image Synthesis
• Factorized Adversarial Networks for Unsupervised Domain Adaptation
• Interpretable Neuron Structuring with Graph Spectral Regularization
• Can GAN Learn Topological Features of a Graph?
• Image Quality Assessment Techniques Show Improved Training and Evaluation of Autoencoder Generative Adversarial Networks
• BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN
• Understanding Human Behaviors in Crowds by Imitating the Decision-Making Process
• Learning a low dimensional manifold of real cancer tissue with PathologyGAN
• Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference
• Formalized Conceptual Spaces with a Geometric Representation of Correlations
• On the Diversity of Realistic Image Synthesis
• REX: Revisiting Budgeted Training with an Improved Schedule
• Unpaired Photo-to-Caricature Translation on Faces in the Wild
• SCH-GAN: Semi-supervised Cross-modal Hashing by Generative Adversarial Network
• The Style-Content Duality of Attractiveness: Learning to Write Eye-Catching Headlines via Disentanglement
• Disentangling Content and Style via Unsupervised Geometry Distillation
• Unsupervised and interpretable scene discovery with Discrete-Attend-Infer-Repeat
• Improving Robot-Centric Learning from Demonstration via Personalized Embeddings
• I Want This Product but Different : Multimodal Retrieval with Synthetic Query Expansion
• Capturing Variabilities from Computed Tomography Images with Generative Adversarial Networks
• Diversifying Semantic Image Synthesis and Editing via Class- and Layer-wise VAEs
• Data Augmentation for Enhancing EEG-based Emotion Recognition with Deep Generative Models
• Hyperprior Induced Unsupervised Disentanglement of Latent Representations
• Disentangling representations of retinal images with generative models
• MosAIc: Finding Artistic Connections across Culture with Conditional Image Retrieval
• Identity-Based Patterns in Deep Convolutional Networks: Generative Adversarial Phonology and Reduplication
• Using Latent Codes for Class Imbalance Problem in Unsupervised Domain Adaptation
• Generative Mixture of Networks
• Pipeline Generative Adversarial Networks for Facial Images Generation with Multiple Attributes
• Depth Structure Preserving Scene Image Generation
• Controlling biases and diversity in diverse image-to-image translation
• Learning Disentangled Representations of Satellite Image Time Series
• Segmentation Guided Image-to-Image Translation with Adversarial Networks
• GANs with Variational Entropy Regularizers: Applications in Mitigating the Mode-Collapse Issue
• Analytically Tractable Inference in Deep Neural Networks
• On generating parametrised structural data using conditional generative adversarial networks
• Image-based Plant Disease Diagnosis with Unsupervised Anomaly Detection Based on Reconstructability of Colors
• Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991)
• Rare Event Detection using Disentangled Representation Learning
• 3D Shape Reconstruction from a Single 2D Image via 2D-3D Self-Consistency
• Class-Distinct and Class-Mutual Image Generation with GANs
• Towards Learning Fine-Grained Disentangled Representations from Speech
• Cross-Domain Adversarial Auto-Encoder
• Lip Movements Generation at a Glance
• Longitudinal Face Aging in the Wild - Recent Deep Learning Approaches
• Natural data structure extracted from neighborhood-similarity graphs
• EVGen: Adversarial Networks for Learning Electric Vehicle Charging Loads and Hidden Representations
• Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification
• Channel-Recurrent Autoencoding for Image Modeling
• Learning to Transfer Graph Embeddings for Inductive Graph based Recommendation
• Generalization to Novel Objects using Prior Relational Knowledge
• SLGAN: Style- and Latent-guided Generative Adversarial Network for Desirable Makeup Transfer and Removal
• Towards a continuous modeling of natural language domains
• Emulating malware authors for proactive protection using GANs over a distributed image visualization of dynamic file behavior
• "Best-of-Many-Samples" Distribution Matching
• How good is my GAN?
• Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey
• Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depth
• Learning Controllable Disentangled Representations with Decorrelation Regularization
• Parametric Synthesis of Text on Stylized Backgrounds using PGGANs
• Controlling Text Edition by Changing Answers of Specific Questions
• Hindsight Value Function for Variance Reduction in Stochastic Dynamic Environment
• Inverting Variational Autoencoders for Improved Generative Accuracy
• Semi-supervised mp-MRI Data Synthesis with StitchLayer and Auxiliary Distance Maximization
• GANE: A Generative Adversarial Network Embedding
• Transferable Natural Language Interface to Structured Queries aided by Adversarial Generation
• TunaGAN: Interpretable GAN for Smart Editing
• Deep Transformation-Invariant Clustering
• Adversarial Learning for Image Forensics Deep Matching with Atrous Convolution
• ChainGAN: A sequential approach to GANs
• Strategic Classification Made Practical
• DPR-CAE: Capsule Autoencoder with Dynamic Part Representation for Image Parsing
• Unsupervised Learning of Neurosymbolic Encoders
• An Introduction to Variational Inference
• Affine Disentangled GAN for Interpretable and Robust AV Perception
• Unsupervised Transformation Learning via Convex Relaxations
• Restricting Greed in Training of Generative Adversarial Network
• Defo-Net: Learning Body Deformation using Generative Adversarial Networks
• Generative Creativity: Adversarial Learning for Bionic Design
• Student's t-Generative Adversarial Networks
• Part-based approximations for morphological operators using asymmetric auto-encoders
• High Diversity Attribute Guided Face Generation with GANs
• Latent Transformations for Object View Points Synthesis
• Grounded Language Understanding for Manipulation Instructions Using GAN-Based Classification
• Escaping from Collapsing Modes in a Constrained Space
• A study on the use of Boundary Equilibrium GAN for Approximate Frontalization of Unconstrained Faces to aid in Surveillance
• Shape-conditioned Image Generation by Learning Latent Appearance Representation from Unpaired Data
• Traversing Latent Space using Decision Ferns
• DEFT: Distilling Entangled Factors by Preventing Information Diffusion
• When Stability meets Sufficiency: Informative Explanations that do not Overwhelm
• Boxhead: A Dataset for Learning Hierarchical Representations
• Moreau-Yoshida Variational Transport: A General Framework For Solving Regularized Distributional Optimization Problems
• A Two-Step Disentanglement Method
• An Efficient Style Virtual Try on Network for Clothing Business Industry
• Cascading Modular Network (CAM-Net) for Multimodal Image Synthesis
• On Study of Mutual Information and its Estimation Methods
• Force-in-domain GAN inversion
• Lifelong Learning Process: Self-Memory Supervising and Dynamically Growing Networks
• DiverseNet: When One Right Answer is not Enough
• Unconstrained Road Marking Recognition with Generative Adversarial Networks
• Exploiting video sequences for unsupervised disentangling in generative adversarial networks
• From abstract items to latent spaces to observed data and back: Compositional Variational Auto-Encoder
• Virtual Conditional Generative Adversarial Networks
• Learning to Clean: A GAN Perspective
• TzK: Flow-Based Conditional Generative Model
• Image Semantic Transformation: Faster, Lighter and Stronger
• Generative Adversarial Networks for Multimodal Representation Learning in Video Hyperlinking
• A Study of Cross-domain Generative Models applied to Cartoon Series
• Instance Map based Image Synthesis with a Denoising Generative Adversarial Network
• Generating a Fusion Image: One's Identity and Another's Shape