Dual Swap Disentangling
arXiv:1805.10583
Abstract
Learning interpretable disentangled representations is a crucial yet challenging task. In this paper, we propose a weakly semi-supervised method, termed as Dual Swap Disentangling (DSD), for disentangling using both labeled and unlabeled data. Unlike conventional weakly supervised methods that rely on full annotations on the group of samples, we require only limited annotations on paired samples that indicate their shared attribute like the color. Our model takes the form of a dual autoencoder structure. To achieve disentangling using the labeled pairs, we follow a "encoding-swap-decoding" process, where we first swap the parts of their encodings corresponding to the shared attribute and then decode the obtained hybrid codes to reconstruct the original input pairs. For unlabeled pairs, we follow the "encoding-swap-decoding" process twice on designated encoding parts and enforce the final outputs to approximate the input pairs. By isolating parts of the encoding and swapping them back and forth, we impose the dimension-wise modularity and portability of the encodings of the unlabeled samples, which implicitly encourages disentangling under the guidance of labeled pairs. This dual swap mechanism, tailored for semi-supervised setting, turns out to be very effective. Experiments on image datasets from a wide domain show that our model yields state-of-the-art disentangling performances.
Accepted by NeurIPS 2018; Adding the theoretical proof for the disentanglement of labeled pairs
References in corpus (10)
- Semi-Supervised Learning with Deep Generative Models
- Deep Convolutional Inverse Graphics Network
- Disentangling by Factorising
- Invertible Conditional GANs for image editing
- Isolating Sources of Disentanglement in Variational Autoencoders
- Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
- DNA-GAN: Learning Disentangled Representations from Multi-Attribute Images
- Learning Disentangled Joint Continuous and Discrete Representations
- Auto-Encoding Total Correlation Explanation
- JADE: Joint Autoencoders for Dis-Entanglement