Competitive Training of Mixtures of Independent Deep Generative Models

Abstract

A common assumption in causal modeling posits that the data is generated by a set of independent mechanisms, and algorithms should aim to recover this structure. Standard unsupervised learning, however, is often concerned with training a single model to capture the overall distribution or aspects thereof. Inspired by clustering approaches, we consider mixtures of implicit generative models that ``disentangle'' the independent generative mechanisms underlying the data. Relying on an additional set of discriminators, we propose a competitive training procedure in which the models only need to capture the portion of the data distribution from which they can produce realistic samples. As a by-product, each model is simpler and faster to train. We empirically show that our approach splits the training distribution in a sensible way and increases the quality of the generated samples.

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Spectral Normalization for Generative Adversarial Networks
• Neural Discrete Representation Learning
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Disentangling by Factorising
• Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
• Learning Independent Causal Mechanisms
• Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering
• CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training
• Multi-Generator Generative Adversarial Nets
• Boosting Variational Inference: an Optimization Perspective
• Boosting Black Box Variational Inference
• An Information-Theoretic Analysis of Hard and Soft Assignment Methods for Clustering

Cited by in corpus (9)

• Object-Centric Learning with Slot Attention
• Causality for Machine Learning
• Are Disentangled Representations Helpful for Abstract Visual Reasoning?
• Weakly-Supervised Disentanglement Without Compromises
• Disentangling Factors of Variation Using Few Labels
• Towards causal generative scene models via competition of experts
• On the Fairness of Disentangled Representations
• Generalization and Robustness Implications in Object-Centric Learning
• Towards Better Understanding of Disentangled Representations via Mutual Information