Composable Generative Models

Abstract

Generative modeling has recently seen many exciting developments with the advent of deep generative architectures such as Variational Auto-Encoders (VAE) or Generative Adversarial Networks (GAN). The ability to draw synthetic i.i.d. observations with the same joint probability distribution as a given dataset has a wide range of applications including representation learning, compression or imputation. It appears that it also has many applications in privacy preserving data analysis, especially when used in conjunction with differential privacy techniques. This paper focuses on synthetic data generation models with privacy preserving applications in mind. It introduces a novel architecture, the Composable Generative Model (CGM) that is state-of-the-art in tabular data generation. Any conditional generative model can be used as a sub-component of the CGM, including CGMs themselves, allowing the generation of numerical, categorical data as well as images, text, or time series. The CGM has been evaluated on 13 datasets (6 standard datasets and 7 simulated) and compared to 14 recent generative models. It beats the state of the art in tabular data generation by a significant margin.

arXiv comment11 pages

References in corpus (8)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Deep Learning with Differential Privacy
• Language Models are Few-Shot Learners
• Data Synthesis based on Generative Adversarial Networks
• Generating Long Sequences with Sparse Transformers
• Generating Multi-label Discrete Patient Records using Generative Adversarial Networks
• Multimodal Generative Models for Scalable Weakly-Supervised Learning
• MHVAE: a Human-Inspired Deep Hierarchical Generative Model for Multimodal Representation Learning