Unsupervised Model Selection for Variational Disentangled Representation Learning

Abstract

Disentangled representations have recently been shown to improve fairness, data efficiency and generalisation in simple supervised and reinforcement learning tasks. To extend the benefits of disentangled representations to more complex domains and practical applications, it is important to enable hyperparameter tuning and model selection of existing unsupervised approaches without requiring access to ground truth attribute labels, which are not available for most datasets. This paper addresses this problem by introducing a simple yet robust and reliable method for unsupervised disentangled model selection. Our approach, Unsupervised Disentanglement Ranking (UDR), leverages the recent theoretical results that explain why variational autoencoders disentangle (Rolinek et al, 2019), to quantify the quality of disentanglement by performing pairwise comparisons between trained model representations. We show that our approach performs comparably to the existing supervised alternatives across 5,400 models from six state of the art unsupervised disentangled representation learning model classes. Furthermore, we show that the ranking produced by our approach correlates well with the final task performance on two different domains.

References in corpus (21)

• Auto-Encoding Variational Bayes
• Practical Bayesian Optimization of Machine Learning Algorithms
• WaveNet: A Generative Model for Raw Audio
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Disentangling by Factorising
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Deep Learning: A Critical Appraisal
• Understanding disentangling in $\beta$-VAE
• Phrase-Based & Neural Unsupervised Machine Translation
• MONet: Unsupervised Scene Decomposition and Representation
• Towards Deep Symbolic Reinforcement Learning
• Convergent Learning: Do different neural networks learn the same representations?
• DARLA: Improving Zero-Shot Transfer in Reinforcement Learning
• Learning Deep Disentangled Embeddings with the F-Statistic Loss
• Are Disentangled Representations Helpful for Abstract Visual Reasoning?
• Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies
• Curiosity Driven Exploration of Learned Disentangled Goal Spaces
• COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration
• Insights on representational similarity in neural networks with canonical correlation
• Improving Generalization for Abstract Reasoning Tasks Using Disentangled Feature Representations

Cited by in corpus (10)

• Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons
• Learning Disentangled Representations in the Imaging Domain
• Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
• Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA
• Disentangling by Subspace Diffusion
• Failure Modes of Variational Autoencoders and Their Effects on Downstream Tasks
• A Large-scale Study on Unsupervised Outlier Model Selection: Do Internal Strategies Suffice?
• Odd-One-Out Representation Learning
• SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision