Learning Conditional Invariance through Cycle Consistency
arXiv:2111.13185 · doi:10.1007/978-3-030-92659-5_24
Abstract
Identifying meaningful and independent factors of variation in a dataset is a challenging learning task frequently addressed by means of deep latent variable models. This task can be viewed as learning symmetry transformations preserving the value of a chosen property along latent dimensions. However, existing approaches exhibit severe drawbacks in enforcing the invariance property in the latent space. We address these shortcomings with a novel approach to cycle consistency. Our method involves two separate latent subspaces for the target property and the remaining input information, respectively. In order to enforce invariance as well as sparsity in the latent space, we incorporate semantic knowledge by using cycle consistency constraints relying on property side information. The proposed method is based on the deep information bottleneck and, in contrast to other approaches, allows using continuous target properties and provides inherent model selection capabilities. We demonstrate on synthetic and molecular data that our approach identifies more meaningful factors which lead to sparser and more interpretable models with improved invariance properties.
arXiv comment16 pages, 3 figures, published at the DAGM German Conference on Pattern Recognition, Sep. 28 - Oct. 1, 2021
References in corpus (16)
- Auto-Encoding Variational Bayes
- Automatic chemical design using a data-driven continuous representation of molecules
- The information bottleneck method
- Semi-Supervised Learning with Deep Generative Models
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Disentangling by Factorising
- Causal Effect Inference with Deep Latent-Variable Models
- Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
- Learning Latent Subspaces in Variational Autoencoders
- InfoGAN-CR and ModelCentrality: Self-supervised Model Training and Selection for Disentangling GANs
- 3DMolNet: A Generative Network for Molecular Structures
- Adversarial Information Factorization
- On the Difference Between the Information Bottleneck and the Deep Information Bottleneck
- Causal Compression
- DualDis: Dual-Branch Disentangling with Adversarial Learning
- Causal learning with sufficient statistics: an information bottleneck approach