Learning Disentangled Representations in the Imaging Domain

Abstract

Disentangled representation learning has been proposed as an approach to learning general representations even in the absence of, or with limited, supervision. A good general representation can be fine-tuned for new target tasks using modest amounts of data, or used directly in unseen domains achieving remarkable performance in the corresponding task. This alleviation of the data and annotation requirements offers tantalising prospects for applications in computer vision and healthcare. In this tutorial paper, we motivate the need for disentangled representations, revisit key concepts, and describe practical building blocks and criteria for learning such representations. We survey applications in medical imaging emphasising choices made in exemplar key works, and then discuss links to computer vision applications. We conclude by presenting limitations, challenges, and opportunities.

arXiv commentAccepted by Medical Image Analysis. This paper follows a tutorial style but also surveys a considerable (more than 260 citations) number of works

References in corpus (73)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• A Simple Framework for Contrastive Learning of Visual Representations
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Bootstrap your own latent: A new approach to self-supervised Learning
• Categorical Reparameterization with Gumbel-Softmax
• The Future of Digital Health with Federated Learning
• Neural Discrete Representation Learning
• Unsupervised Domain Adaptation by Backpropagation
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• The information bottleneck method
• Measuring and testing dependence by correlation of distances
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• Normalizing Flows: An Introduction and Review of Current Methods
• Toward Multimodal Image-to-Image Translation
• CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Disentangling by Factorising
• Pyro: Deep Universal Probabilistic Programming
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Group Equivariant Convolutional Networks
• Multivariate mixture model for myocardium segmentation combining multi-source images
• Causality matters in medical imaging
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• Towards a Definition of Disentangled Representations
• Understanding disentangling in $\beta$-VAE
• Towards Out-Of-Distribution Generalization: A Survey
• ADN: Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction
• Disentangled Representation Learning in Cardiac Image Analysis
• Learning Disentangled Representations with Semi-Supervised Deep Generative Models
• Discovering Hidden Factors of Variation in Deep Networks
• Variational Autoencoders and Nonlinear ICA: A Unifying Framework
• Learning Deep Disentangled Embeddings with the F-Statistic Loss
• Structured Disentangled Representations
• Generalizing to Unseen Domains: A Survey on Domain Generalization
• Deep Structural Causal Models for Tractable Counterfactual Inference
• Towards Causal Representation Learning
• ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching
• Disentangle, align and fuse for multimodal and semi-supervised image segmentation
• Unsupervised Multi-modal Style Transfer for Cardiac MR Segmentation
• Closed-Form Factorization of Latent Semantics in GANs
• Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style
• Pseudo-healthy synthesis with pathology disentanglement and adversarial learning
• GeneGAN: Learning Object Transfiguration and Attribute Subspace from Unpaired Data
• Independently Controllable Features
• On Disentangled Representations Learned From Correlated Data
• Unsupervised Model Selection for Variational Disentangled Representation Learning
• Conditional Generation of Medical Images via Disentangled Adversarial Inference
• Deep Learning based Multi-modal Computing with Feature Disentanglement for MRI Image Synthesis
• Interpretable Deep Models for Cardiac Resynchronisation Therapy Response Prediction
• Decompose to manipulate: Manipulable Object Synthesis in 3D Medical Images with Structured Image Decomposition
• Medical Image Harmonization Using Deep Learning Based Canonical Mapping: Toward Robust and Generalizable Learning in Imaging
• Symmetry-Based Disentangled Representation Learning requires Interaction with Environments
• Harmonic Unpaired Image-to-image Translation
• Learning Disentangled Representation by Exploiting Pretrained Generative Models: A Contrastive Learning View
• Measuring Disentanglement: A Review of Metrics
• Adversarial Pseudo Healthy Synthesis Needs Pathology Factorization
• GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images
• Disentangling Influence: Using Disentangled Representations to Audit Model Predictions
• Information-based Disentangled Representation Learning for Unsupervised MR Harmonization
• Order-Guided Disentangled Representation Learning for Ulcerative Colitis Classification with Limited Labels
• Bidirectional Generative Modeling Using Adversarial Gradient Estimation
• Measuring the Biases and Effectiveness of Content-Style Disentanglement
• Measuring Unintended Memorisation of Unique Private Features in Neural Networks
• Evaluating and Mitigating Bias in Image Classifiers: A Causal Perspective Using Counterfactuals
• Explainable-by-design Semi-Supervised Representation Learning for COVID-19 Diagnosis from CT Imaging
• Lung Swapping Autoencoder: Learning a Disentangled Structure-texture Representation of Chest Radiographs
• A Deep Network for Joint Registration and Reconstruction of Images with Pathologies
• Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval
• Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction
• Evaluating Disentanglement of Structured Representations

Cited by in corpus (4)

• A survey of recent methods for addressing AI fairness and bias in biomedicine
• SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI
• Disentangling representations of retinal images with generative models
• Approaching an unknown communication system by latent space exploration and causal inference