Decoupling Global and Local Representations via Invertible Generative Flows

Abstract

In this work, we propose a new generative model that is capable of automatically decoupling global and local representations of images in an entirely unsupervised setting, by embedding a generative flow in the VAE framework to model the decoder. Specifically, the proposed model utilizes the variational auto-encoding framework to learn a (low-dimensional) vector of latent variables to capture the global information of an image, which is fed as a conditional input to a flow-based invertible decoder with architecture borrowed from style transfer literature. Experimental results on standard image benchmarks demonstrate the effectiveness of our model in terms of density estimation, image generation and unsupervised representation learning. Importantly, this work demonstrates that with only architectural inductive biases, a generative model with a likelihood-based objective is capable of learning decoupled representations, requiring no explicit supervision. The code for our model is available at https://github.com/XuezheMax/wolf.

arXiv commentCamera-ready at ICLR 2021. 23 pages (plus appendix), 16 figures, 5 tables. Due to arxiv size constraints, this version is using downscaled images. Please download the full-resolution version from https://vixra.org/abs/2004.0222

References in corpus (26)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Representation Learning with Contrastive Predictive Coding
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Learning deep representations by mutual information estimation and maximization
• Pixel Recurrent Neural Networks
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• Generative Modeling by Estimating Gradients of the Data Distribution
• Density estimation using Real NVP
• Disentangling by Factorising
• Variational Lossy Autoencoder
• Disentangling factors of variation in deep representations using adversarial training
• Residual Flows for Invertible Generative Modeling
• RNADE: The real-valued neural autoregressive density-estimator
• Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling
• Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models
• VFlow: More Expressive Generative Flows with Variational Data Augmentation
• A RAD approach to deep mixture models
• Variational Autoencoders with Normalizing Flow Decoders
• MAE: Mutual Posterior-Divergence Regularization for Variational AutoEncoders
• Learning Disentangled Representations via Mutual Information Estimation
• Latent Normalizing Flows for Many-to-Many Cross-Domain Mappings
• Deep Variational Inference Without Pixel-Wise Reconstruction