Desiderata for Representation Learning: A Causal Perspective
arXiv:2109.03795
Abstract
Representation learning constructs low-dimensional representations to summarize essential features of high-dimensional data. This learning problem is often approached by describing various desiderata associated with learned representations; e.g., that they be non-spurious, efficient, or disentangled. It can be challenging, however, to turn these intuitive desiderata into formal criteria that can be measured and enhanced based on observed data. In this paper, we take a causal perspective on representation learning, formalizing non-spuriousness and efficiency (in supervised representation learning) and disentanglement (in unsupervised representation learning) using counterfactual quantities and observable consequences of causal assertions. This yields computable metrics that can be used to assess the degree to which representations satisfy the desiderata of interest and learn non-spurious and disentangled representations from single observational datasets.
arXiv comment68 pages
References in corpus (29)
- Auto-Encoding Variational Bayes
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Disentangling by Factorising
- Underspecification Presents Challenges for Credibility in Modern Machine Learning
- Invariant Risk Minimization
- Frequentist Consistency of Variational Bayes
- On Learning Invariant Representation for Domain Adaptation
- Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
- Invariant Representations without Adversarial Training
- Weakly-Supervised Disentanglement Without Compromises
- Learning Independent Causal Mechanisms
- Weakly Supervised Disentanglement with Guarantees
- Adapting Text Embeddings for Causal Inference
- Independently Controllable Factors
- Disentangling Factors of Variation Using Few Labels
- Disentangling the independently controllable factors of variation by interacting with the world
- Representation Learning via Invariant Causal Mechanisms
- On Disentangled Representations Learned From Correlated Data
- Graph-based Isometry Invariant Representation Learning
- Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects
- Multiple Causal Inference with Latent Confounding
- Using Embeddings to Correct for Unobserved Confounding in Networks
- Generalization in anti-causal learning
- Towards Clarifying the Theory of the Deconfounder
- Invariant Representation Learning for Treatment Effect Estimation
- Disentangled Representation Learning with Wasserstein Total Correlation
- Deep causal representation learning for unsupervised domain adaptation
- Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests
- Intact-VAE: Estimating Treatment Effects under Unobserved Confounding