Adversarially-learned Inference via an Ensemble of Discrete Undirected Graphical Models

Abstract

Undirected graphical models are compact representations of joint probability distributions over random variables. To solve inference tasks of interest, graphical models of arbitrary topology can be trained using empirical risk minimization. However, to solve inference tasks that were not seen during training, these models (EGMs) often need to be re-trained. Instead, we propose an inference-agnostic adversarial training framework which produces an infinitely-large ensemble of graphical models (AGMs). The ensemble is optimized to generate data within the GAN framework, and inference is performed using a finite subset of these models. AGMs perform comparably with EGMs on inference tasks that the latter were specifically optimized for. Most importantly, AGMs show significantly better generalization to unseen inference tasks compared to EGMs, as well as deep neural architectures like GibbsNet and VAEAC which allow arbitrary conditioning. Finally, AGMs allow fast data sampling, competitive with Gibbs sampling from EGMs.

arXiv comment17 pages, 5 figures, 5 tables. NeurIPS 2020

References in corpus (25)

• Distilling the Knowledge in a Neural Network
• Conditional Generative Adversarial Nets
• Neural Message Passing for Quantum Chemistry
• Categorical Reparameterization with Gumbel-Softmax
• Conditional Random Fields as Recurrent Neural Networks
• Adversarially Learned Inference
• Generating Multi-label Discrete Patient Records using Generative Adversarial Networks
• Learning feed-forward one-shot learners
• Composing graphical models with neural networks for structured representations and fast inference
• NetGAN: Generating Graphs via Random Walks
• GraphGAN: Graph Representation Learning with Generative Adversarial Nets
• GMNN: Graph Markov Neural Networks
• Learning Graphical Model Parameters with Approximate Marginal Inference
• Neural Belief-Propagation Decoders for Quantum Error-Correcting Codes
• Modeling Tabular data using Conditional GAN
• Learning Approximate Inference Networks for Structured Prediction
• Generating Multi-Categorical Samples with Generative Adversarial Networks
• Neural Enhanced Belief Propagation on Factor Graphs
• Graphical Generative Adversarial Networks
• Adversarial Message Passing For Graphical Models
• Factor Graph Neural Network
• A Universal Marginalizer for Amortized Inference in Generative Models
• Training Generative Adversarial Networks with Binary Neurons by End-to-end Backpropagation
• Generative Adversarial Networks and Probabilistic Graph Models for Hyperspectral Image Classification
• GibbsNet: Iterative Adversarial Inference for Deep Graphical Models