Variational Inference of Disentangled Latent Concepts from Unlabeled Observations

Abstract

Disentangled representations, where the higher level data generative factors are reflected in disjoint latent dimensions, offer several benefits such as ease of deriving invariant representations, transferability to other tasks, interpretability, etc. We consider the problem of unsupervised learning of disentangled representations from large pool of unlabeled observations, and propose a variational inference based approach to infer disentangled latent factors. We introduce a regularizer on the expectation of the approximate posterior over observed data that encourages the disentanglement. We also propose a new disentanglement metric which is better aligned with the qualitative disentanglement observed in the decoder's output. We empirically observe significant improvement over existing methods in terms of both disentanglement and data likelihood (reconstruction quality).

ICLR 2018 Version

References in corpus (9)

• Deep Convolutional Inverse Graphics Network
• Generative Moment Matching Networks
• Discovering Hidden Factors of Variation in Deep Networks
• Towards Deeper Understanding of Variational Autoencoding Models
• A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
• On integral probability metrics, ϕ-divergences and binary classification
• Disentangling Factors of Variation via Generative Entangling
• A Survey of Inductive Biases for Factorial Representation-Learning
• Local Group Invariant Representations via Orbit Embeddings

Cited by in corpus (9)

• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling
• Identifying through Flows for Recovering Latent Representations
• Linear Disentangled Representations and Unsupervised Action Estimation
• Geodesics in fibered latent spaces: A geometric approach to learning correspondences between conditions
• Boxhead: A Dataset for Learning Hierarchical Representations
• DEFT: Distilling Entangled Factors by Preventing Information Diffusion
• Divide and Rule: Recurrent Partitioned Network for Dynamic Processes