Weakly-Supervised Disentanglement Without Compromises

Abstract

Intelligent agents should be able to learn useful representations by observing changes in their environment. We model such observations as pairs of non-i.i.d. images sharing at least one of the underlying factors of variation. First, we theoretically show that only knowing how many factors have changed, but not which ones, is sufficient to learn disentangled representations. Second, we provide practical algorithms that learn disentangled representations from pairs of images without requiring annotation of groups, individual factors, or the number of factors that have changed. Third, we perform a large-scale empirical study and show that such pairs of observations are sufficient to reliably learn disentangled representations on several benchmark data sets. Finally, we evaluate our learned representations and find that they are simultaneously useful on a diverse suite of tasks, including generalization under covariate shifts, fairness, and abstract reasoning. Overall, our results demonstrate that weak supervision enables learning of useful disentangled representations in realistic scenarios.

arXiv commentWe updated the description of the generation of the dataset compared to the ICML version

References in corpus (32)

• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Deep Convolutional Inverse Graphics Network
• Disentangling by Factorising
• Discovering physical concepts with neural networks
• Recent Advances in Autoencoder-Based Representation Learning
• Towards a Definition of Disentangled Representations
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Disentangling factors of variation in deep representations using adversarial training
• Disentangled Representation Learning in Cardiac Image Analysis
• Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
• Measuring abstract reasoning in neural networks
• Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
• Discovering Hidden Factors of Variation in Deep Networks
• Flexibly Fair Representation Learning by Disentanglement
• On Causal and Anticausal Learning
• DARLA: Improving Zero-Shot Transfer in Reinforcement Learning
• A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
• Variational Autoencoders and Nonlinear ICA: A Unifying Framework
• Causality for Machine Learning
• Learning Deep Disentangled Embeddings with the F-Statistic Loss
• A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
• Are Disentangled Representations Helpful for Abstract Visual Reasoning?
• The Consciousness Prior
• On the relation between accuracy and fairness in binary classification
• Disentangled Sequential Autoencoder
• Weakly Supervised Disentanglement with Guarantees
• Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)
• Disentangling the independently controllable factors of variation by interacting with the world
• A Survey of Inductive Biases for Factorial Representation-Learning
• Learning to Linearize Under Uncertainty
• Understanding Visual Concepts with Continuation Learning
• Competitive Training of Mixtures of Independent Deep Generative Models

Cited by in corpus (34)

• Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons
• On Disentangled Representations Learned From Correlated Data
• Contrastive Learning Inverts the Data Generating Process
• Assaying Out-Of-Distribution Generalization in Transfer Learning
• Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
• Desiderata for Representation Learning: A Causal Perspective
• 3D-IDS: Doubly Disentangled Dynamic Intrusion Detection
• Contrastively Disentangled Sequential Variational Autoencoder
• Learning and controlling the source-filter representation of speech with a variational autoencoder
• Unsupervised Learning of Compositional Energy Concepts
• Counterfactual Data Augmentation using Locally Factored Dynamics
• On the Transfer of Disentangled Representations in Realistic Settings
• Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning
• Human-interpretable model explainability on high-dimensional data
• Benchmarks, Algorithms, and Metrics for Hierarchical Disentanglement
• Towards Robust Metrics for Concept Representation Evaluation
• Visual Representation Learning Does Not Generalize Strongly Within the Same Domain
• An Improved Semi-Supervised VAE for Learning Disentangled Representations
• Addressing the Topological Defects of Disentanglement via Distributed Operators
• DynamicVAE: Decoupling Reconstruction Error and Disentangled Representation Learning
• On Disentanglement in Gaussian Process Variational Autoencoders
• Video Reenactment as Inductive Bias for Content-Motion Disentanglement
• Grounding inductive biases in natural images:invariance stems from variations in data
• On the Transferability of VAE Embeddings using Relational Knowledge with Semi-Supervision
• Deep Anomaly Detection by Residual Adaptation
• Odd-One-Out Representation Learning
• Amortized backward variational inference in nonlinear state-space models
• Disentanglement of Latent Representations via Causal Interventions
• Mixture Representation Learning with Coupled Autoencoders
• Group-disentangled Representation Learning with Weakly-Supervised Regularization
• Robust Disentanglement of a Few Factors at a Time
• Boxhead: A Dataset for Learning Hierarchical Representations
• DAReN: A Collaborative Approach Towards Reasoning And Disentangling
• Proceedings of the First Workshop on Weakly Supervised Learning (WeaSuL)