Learning Disentangled Representations with Reference-Based Variational Autoencoders

Abstract

Learning disentangled representations from visual data, where different high-level generative factors are independently encoded, is of importance for many computer vision tasks. Solving this problem, however, typically requires to explicitly label all the factors of interest in training images. To alleviate the annotation cost, we introduce a learning setting which we refer to as "reference-based disentangling". Given a pool of unlabeled images, the goal is to learn a representation where a set of target factors are disentangled from others. The only supervision comes from an auxiliary "reference set" containing images where the factors of interest are constant. In order to address this problem, we propose reference-based variational autoencoders, a novel deep generative model designed to exploit the weak-supervision provided by the reference set. By addressing tasks such as feature learning, conditional image generation or attribute transfer, we validate the ability of the proposed model to learn disentangled representations from this minimal form of supervision.

References in corpus (13)

• Auto-Encoding Variational Bayes
• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• Semi-Supervised Learning with Deep Generative Models
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Deep Convolutional Inverse Graphics Network
• Disentangling by Factorising
• Decomposing Motion and Content for Natural Video Sequence Prediction
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Disentangling factors of variation in deep representations using adversarial training
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
• Disentangling Factors of Variation via Generative Entangling
• Variational Inference using Implicit Distributions

Cited by in corpus (7)

• Are Disentangled Representations Helpful for Abstract Visual Reasoning?
• Disentangling Factors of Variation Using Few Labels
• On the Fairness of Disentangled Representations
• Product of Orthogonal Spheres Parameterization for Disentangled Representation Learning
• DualDis: Dual-Branch Disentangling with Adversarial Learning
• WeLa-VAE: Learning Alternative Disentangled Representations Using Weak Labels
• SepVAE: a contrastive VAE to separate pathological patterns from healthy ones