Mixture Representation Learning with Coupled Autoencoders

Abstract

Jointly identifying a mixture of discrete and continuous factors of variability without supervision is a key problem in unraveling complex phenomena. Variational inference has emerged as a promising method to learn interpretable mixture representations. However, posterior approximation in high-dimensional latent spaces, particularly for discrete factors remains challenging. Here, we propose an unsupervised variational framework using multiple interacting networks called cpl-mixVAE that scales well to high-dimensional discrete settings. In this framework, the mixture representation of each network is regularized by imposing a consensus constraint on the discrete factor. We justify the use of this framework by providing both theoretical and experimental results. Finally, we use the proposed method to jointly uncover discrete and continuous factors of variability describing gene expression in a single-cell transcriptomic dataset profiling more than a hundred cortical neuron types.

arXiv comment10 pages, 6 figures, conference

References in corpus (19)

• Auto-Encoding Variational Bayes
• Categorical Reparameterization with Gumbel-Softmax
• Data Augmentation Generative Adversarial Networks
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Disentangling by Factorising
• Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
• Regularizing Neural Networks by Penalizing Confident Output Distributions
• Recent Advances in Autoencoder-Based Representation Learning
• Understanding disentangling in $\beta$-VAE
• Improving Variational Inference with Inverse Autoregressive Flow
• Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
• Weakly-Supervised Disentanglement Without Compromises
• Weakly Supervised Disentanglement with Guarantees
• Boosting Variational Inference
• Learning Discrete and Continuous Factors of Data via Alternating Disentanglement
• Gaussian variational approximation for high-dimensional state space models
• A coupled autoencoder approach for multi-modal analysis of cell types
• Boosting Black Box Variational Inference
• Private-Shared Disentangled Multimodal VAE for Learning of Hybrid Latent Representations