GANs with Conditional Independence Graphs: On Subadditivity of Probability Divergences

Abstract

Generative Adversarial Networks (GANs) are modern methods to learn the underlying distribution of a data set. GANs have been widely used in sample synthesis, de-noising, domain transfer, etc. GANs, however, are designed in a model-free fashion where no additional information about the underlying distribution is available. In many applications, however, practitioners have access to the underlying independence graph of the variables, either as a Bayesian network or a Markov Random Field (MRF). We ask: how can one use this additional information in designing model-based GANs? In this paper, we provide theoretical foundations to answer this question by studying subadditivity properties of probability divergences, which establish upper bounds on the distance between two high-dimensional distributions by the sum of distances between their marginals over (local) neighborhoods of the graphical structure of the Bayes-net or the MRF. We prove that several popular probability divergences satisfy some notion of subadditivity under mild conditions. These results lead to a principled design of a model-based GAN that uses a set of simple discriminators on the neighborhoods of the Bayes-net/MRF, rather than a giant discriminator on the entire network, providing significant statistical and computational benefits. Our experiments on synthetic and real-world datasets demonstrate the benefits of our principled design of model-based GANs.

References in corpus (13)

• WaveNet: A Generative Model for Raw Audio
• Categorical Reparameterization with Gumbel-Softmax
• Energy-based Generative Adversarial Network
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• Generative Moment Matching Networks
• $f$-divergence Inequalities
• Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• Patch-Based Image Inpainting with Generative Adversarial Networks
• Adversarial Audio Synthesis
• An efficient linear programming method for Optimal Transportation
• Understanding GANs: the LQG Setting
• Square Hellinger Subadditivity for Bayesian Networks and its Applications to Identity Testing

Cited by in corpus (1)

• Learning latent causal graphs via mixture oracles