OOGAN: Disentangling GAN with One-Hot Sampling and Orthogonal Regularization
arXiv:1905.10836
Abstract
Exploring the potential of GANs for unsupervised disentanglement learning, this paper proposes a novel GAN-based disentanglement framework with One-Hot Sampling and Orthogonal Regularization (OOGAN). While previous works mostly attempt to tackle disentanglement learning through VAE and seek to implicitly minimize the Total Correlation (TC) objective with various sorts of approximation methods, we show that GANs have a natural advantage in disentangling with an alternating latent variable (noise) sampling method that is straightforward and robust. Furthermore, we provide a brand-new perspective on designing the structure of the generator and discriminator, demonstrating that a minor structural change and an orthogonal regularization on model weights entails an improved disentanglement. Instead of experimenting on simple toy datasets, we conduct experiments on higher-resolution images and show that OOGAN greatly pushes the boundary of unsupervised disentanglement.
arXiv commentAAAI 2020
References in corpus (9)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Categorical Reparameterization with Gumbel-Softmax
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Bayesian Compression for Deep Learning
- CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Norms
- Entropy and mutual information in models of deep neural networks
- Structured Disentangled Representations
- Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks