On the Transfer of Disentangled Representations in Realistic Settings
arXiv:2010.14407
Abstract
Learning meaningful representations that disentangle the underlying structure of the data generating process is considered to be of key importance in machine learning. While disentangled representations were found to be useful for diverse tasks such as abstract reasoning and fair classification, their scalability and real-world impact remain questionable. We introduce a new high-resolution dataset with 1M simulated images and over 1,800 annotated real-world images of the same setup. In contrast to previous work, this new dataset exhibits correlations, a complex underlying structure, and allows to evaluate transfer to unseen simulated and real-world settings where the encoder i) remains in distribution or ii) is out of distribution. We propose new architectures in order to scale disentangled representation learning to realistic high-resolution settings and conduct a large-scale empirical study of disentangled representations on this dataset. We observe that disentanglement is a good predictor for out-of-distribution (OOD) task performance.
arXiv commentPublished at ICLR 2021
References in corpus (22)
- Auto-Encoding Variational Bayes
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Disentangling by Factorising
- Domain Generalization via Invariant Feature Representation
- Invariant Risk Minimization
- Out-of-Distribution Generalization via Risk Extrapolation (REx)
- Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
- Invariant Models for Causal Transfer Learning
- ReZero is All You Need: Fast Convergence at Large Depth
- Are Disentangled Representations Helpful for Abstract Visual Reasoning?
- Learning Deep Disentangled Embeddings with the F-Statistic Loss
- Sim-to-Real Robot Learning from Pixels with Progressive Nets
- Weakly-Supervised Disentanglement Without Compromises
- SCAN: Learning Hierarchical Compositional Visual Concepts
- Weakly Supervised Disentanglement with Guarantees
- On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset
- Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)
- On Disentangled Representations Learned From Correlated Data
- TriFinger: An Open-Source Robot for Learning Dexterity
- Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
- How to Close Sim-Real Gap? Transfer with Segmentation!