Challenges in Disentangling Independent Factors of Variation

Abstract

We study the problem of building models that disentangle independent factors of variation. Such models could be used to encode features that can efficiently be used for classification and to transfer attributes between different images in image synthesis. As data we use a weakly labeled training set. Our weak labels indicate what single factor has changed between two data samples, although the relative value of the change is unknown. This labeling is of particular interest as it may be readily available without annotation costs. To make use of weak labels we introduce an autoencoder model and train it through constraints on image pairs and triplets. We formally prove that without additional knowledge there is no guarantee that two images with the same factor of variation will be mapped to the same feature. We call this issue the reference ambiguity. Moreover, we show the role of the feature dimensionality and adversarial training. We demonstrate experimentally that the proposed model can successfully transfer attributes on several datasets, but show also cases when the reference ambiguity occurs.

arXiv commentSubmitted to ICLR 2018

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Adversarial Feature Learning
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Discovering Hidden Factors of Variation in Deep Networks

Cited by in corpus (9)

• Disentangled Representation Learning in Cardiac Image Analysis
• Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
• Guided Variational Autoencoder for Disentanglement Learning
• Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders
• Disentangling Controllable and Uncontrollable Factors of Variation by Interacting with the World
• An Improved Semi-Supervised VAE for Learning Disentangled Representations
• Towards Disentangled Representations for Human Retargeting by Multi-view Learning
• Unsupervised Shape and Pose Disentanglement for 3D Meshes
• Disentangling Pose from Appearance in Monochrome Hand Images