Explicit Disentanglement of Appearance and Perspective in Generative Models
arXiv:1906.11881
Abstract
Disentangled representation learning finds compact, independent and easy-to-interpret factors of the data. Learning such has been shown to require an inductive bias, which we explicitly encode in a generative model of images. Specifically, we propose a model with two latent spaces: one that represents spatial transformations of the input data, and another that represents the transformed data. We find that the latter naturally captures the intrinsic appearance of the data. To realize the generative model, we propose a Variationally Inferred Transformational Autoencoder (VITAE) that incorporates a spatial transformer into a variational autoencoder. We show how to perform inference in the model efficiently by carefully designing the encoders and restricting the transformation class to be diffeomorphic. Empirically, our model separates the visual style from digit type on MNIST, separates shape and pose in images of human bodies and facial features from facial shape on CelebA.
arXiv comment9 main pages + 2 pages references + 8 pages of supplementary material
References in corpus (12)
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Deep Convolutional Inverse Graphics Network
- Towards a Definition of Disentangled Representations
- Improving Variational Inference with Inverse Autoregressive Flow
- Representation Learning: A Review and New Perspectives
- Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning
- ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
- Deformable Generator Networks: Unsupervised Disentanglement of Appearance and Geometry
- Inverse Compositional Spatial Transformer Networks
- DeSTNet: Densely Fused Spatial Transformer Networks