Towards a Definition of Disentangled Representations

Abstract

How can intelligent agents solve a diverse set of tasks in a data-efficient manner? The disentangled representation learning approach posits that such an agent would benefit from separating out (disentangling) the underlying structure of the world into disjoint parts of its representation. However, there is no generally agreed-upon definition of disentangling, not least because it is unclear how to formalise the notion of world structure beyond toy datasets with a known ground truth generative process. Here we propose that a principled solution to characterising disentangled representations can be found by focusing on the transformation properties of the world. In particular, we suggest that those transformations that change only some properties of the underlying world state, while leaving all other properties invariant, are what gives exploitable structure to any kind of data. Similar ideas have already been successfully applied in physics, where the study of symmetry transformations has revolutionised the understanding of the world structure. By connecting symmetry transformations to vector representations using the formalism of group and representation theory we arrive at the first formal definition of disentangled representations. Our new definition is in agreement with many of the current intuitions about disentangling, while also providing principled resolutions to a number of previous points of contention. While this work focuses on formally defining disentangling - as opposed to solving the learning problem - we believe that the shift in perspective to studying data transformations can stimulate the development of better representation learning algorithms.

References in corpus (1)

• Unsupervised Learning of Disentangled Representations from Video

Cited by in corpus (108)

• Multi-Object Representation Learning with Iterative Variational Inference
• Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
• Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons
• Variational Autoencoders and Nonlinear ICA: A Unifying Framework
• Are Disentangled Representations Helpful for Abstract Visual Reasoning?
• Learning Disentangled Representations in the Imaging Domain
• Weakly-Supervised Disentanglement Without Compromises
• DIVA: Domain Invariant Variational Autoencoders
• Weakly Supervised Disentanglement with Guarantees
• Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs
• Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends
• Disentangling Disentanglement in Variational Autoencoders
• Invariant Collaborative Filtering to Popularity Distribution Shift
• On the Binding Problem in Artificial Neural Networks
• Disentangling Factors of Variation Using Few Labels
• Learning Causal Semantic Representation for Out-of-Distribution Prediction
• Designing Ecosystems of Intelligence from First Principles
• Equivariant Hamiltonian Flows
• From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence
• Understanding the Limitations of Variational Mutual Information Estimators
• Exploring particle dynamics during self-organization processes via rotationally invariant latent representations
• Bayesian multiscale deep generative model for the solution of high-dimensional inverse problems
• Diffusion Variational Autoencoders
• Density-aware Haze Image Synthesis by Self-Supervised Content-Style Disentanglement
• Counterfactuals uncover the modular structure of deep generative models
• Hamiltonian Generative Networks
• Unsupervised State Representation Learning in Atari
• Disentangled Variational Autoencoder for Emotion Recognition in Conversations
• Counterfactual Zero-Shot and Open-Set Visual Recognition
• Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness
• Learning Symmetries of Classical Integrable Systems
• Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
• LagNetViP: A Lagrangian Neural Network for Video Prediction
• Generative causal explanations of black-box classifiers
• Symmetry-Based Disentangled Representation Learning requires Interaction with Environments
• Explicit Disentanglement of Appearance and Perspective in Generative Models
• Multi-Scale and Multi-Layer Contrastive Learning for Domain Generalization
• Theory and Evaluation Metrics for Learning Disentangled Representations
• Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity
• Learning and controlling the source-filter representation of speech with a variational autoencoder
• Learning Group Structure and Disentangled Representations of Dynamical Environments
• Learning to Predict Without Looking Ahead: World Models Without Forward Prediction
• Editing in Style: Uncovering the Local Semantics of GANs
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
• Disentangled Representations from Non-Disentangled Models
• Disentangled Representation Learning with Wasserstein Total Correlation
• On the Fairness of Disentangled Representations
• Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations
• Product of Orthogonal Spheres Parameterization for Disentangled Representation Learning
• Disentangling Influence: Using Disentangled Representations to Audit Model Predictions
• Model-Based Domain Generalization
• Orientation-Disentangled Unsupervised Representation Learning for Computational Pathology
• Disentangled Cumulants Help Successor Representations Transfer to New Tasks
• Disentangled Item Representation for Recommender Systems
• DynamicVAE: Decoupling Reconstruction Error and Disentangled Representation Learning
• A Plug-in Method for Representation Factorization in Connectionist Models
• Disentangling by Subspace Diffusion
• Reimagining an autonomous vehicle
• Measuring the Biases and Effectiveness of Content-Style Disentanglement
• Interpretable Neuron Structuring with Graph Spectral Regularization
• Data-efficient visuomotor policy training using reinforcement learning and generative models
• Product Kanerva Machines: Factorized Bayesian Memory
• Weakly-Supervised Reinforcement Learning for Controllable Behavior
• Unsupervised Emergence of Egocentric Spatial Structure from Sensorimotor Prediction
• Disentangling representations of retinal images with generative models
• Downbeat Tracking with Tempo-Invariant Convolutional Neural Networks
• Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy
• Disentangling to Cluster: Gaussian Mixture Variational Ladder Autoencoders
• How to Not Measure Disentanglement
• Auditing AI models for Verified Deployment under Semantic Specifications
• Video Reenactment as Inductive Bias for Content-Motion Disentanglement
• STAN-CT: Standardizing CT Image using Generative Adversarial Network
• Bayes-Factor-VAE: Hierarchical Bayesian Deep Auto-Encoder Models for Factor Disentanglement
• Transporting Causal Mechanisms for Unsupervised Domain Adaptation
• Linear Disentangled Representations and Unsupervised Action Estimation
• Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective
• Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval
• A Metric for Linear Symmetry-Based Disentanglement
• No Representation without Transformation
• Gauge theory and twins paradox of disentangled representations
• Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations
• Disentangled (Un)Controllable Features
• Koopman Q-learning: Offline Reinforcement Learning via Symmetries of Dynamics
• Disentanglement Learning via Topology
• On the Sensory Commutativity of Action Sequences for Embodied Agents
• Disentanglement of Latent Representations via Causal Interventions
• Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning
• Robust Disentanglement of a Few Factors at a Time
• Problems of representation of electrocardiograms in convolutional neural networks
• Understanding Attention: In Minds and Machines
• Joint Estimation of Image Representations and their Lie Invariants
• Formalising Concepts as Grounded Abstractions
• A Deeper Look at the Unsupervised Learning of Disentangled Representations in $\beta$-VAE from the Perspective of Core Object Recognition
• Longitudinal Self-Supervised Learning
• Semantic Network Interpretation
• Deep Variational Luenberger-type Observer for Stochastic Video Prediction
• Kullback-Leibler Divergence-Based Out-of-Distribution Detection with Flow-Based Generative Models
• Quantifying and Learning Linear Symmetry-Based Disentanglement
• DEFT: Distilling Entangled Factors by Preventing Information Diffusion
• Towards Better Understanding of Disentangled Representations via Mutual Information
• Disentangling Representations of Text by Masking Transformers
• The Role of General Intelligence in Mathematical Reasoning
• A moment-matching metric for latent variable generative models
• Quantised Transforming Auto-Encoders: Achieving Equivariance to Arbitrary Transformations in Deep Networks
• Domain-Agnostic Learning with Anatomy-Consistent Embedding for Cross-Modality Liver Segmentation
• Efficiently Disentangle Causal Representations
• Interpretable Deep Learning for Stock Returns: A Consensus-Bottleneck Asset Pricing Model
• Learning to Transfer: A Foliated Theory