Hierarchical Implicit Models and Likelihood-Free Variational Inference

Abstract

Implicit probabilistic models are a flexible class of models defined by a simulation process for data. They form the basis for theories which encompass our understanding of the physical world. Despite this fundamental nature, the use of implicit models remains limited due to challenges in specifying complex latent structure in them, and in performing inferences in such models with large data sets. In this paper, we first introduce hierarchical implicit models (HIMs). HIMs combine the idea of implicit densities with hierarchical Bayesian modeling, thereby defining models via simulators of data with rich hidden structure. Next, we develop likelihood-free variational inference (LFVI), a scalable variational inference algorithm for HIMs. Key to LFVI is specifying a variational family that is also implicit. This matches the model's flexibility and allows for accurate approximation of the posterior. We demonstrate diverse applications: a large-scale physical simulator for predator-prey populations in ecology; a Bayesian generative adversarial network for discrete data; and a deep implicit model for text generation.

Appears in Neural Information Processing Systems, 2017

References in corpus (8)

• Weight Uncertainty in Neural Networks
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Edward: A library for probabilistic modeling, inference, and criticism
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• Learning Stochastic Recurrent Networks
• Learning in Implicit Generative Models
• On distinguishability criteria for estimating generative models
• Adversarial Message Passing For Graphical Models

Cited by in corpus (59)

• Spectral Normalization for Generative Adversarial Networks
• The frontier of simulation-based inference
• cGANs with Projection Discriminator
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• An Overview of Deep Semi-Supervised Learning
• Masking: A New Perspective of Noisy Supervision
• Mining gold from implicit models to improve likelihood-free inference
• Image Augmentations for GAN Training
• Boundary-Seeking Generative Adversarial Networks
• Deep Active Inference
• Deep Probabilistic Programming
• SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects
• Implicit Weight Uncertainty in Neural Networks
• Unsupervised Object Segmentation by Redrawing
• Deep Neural Generative Model of Functional MRI Images for Psychiatric Disorder Diagnosis
• Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation
• Gradient Estimators for Implicit Models
• Consistency Regularization for Generative Adversarial Networks
• A Spectral Approach to Gradient Estimation for Implicit Distributions
• Adversarially Regularized Autoencoders
• Improved Consistency Regularization for GANs
• Approximate Inference with Amortised MCMC
• Semi-Implicit Variational Inference
• Advances in Variational Inference
• Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies
• Effective LHC measurements with matrix elements and machine learning
• Adversarial Variational Optimization of Non-Differentiable Simulators
• Likelihood-free inference with an improved cross-entropy estimator
• The Medical Deconfounder: Assessing Treatment Effects with Electronic Health Records
• Variational Implicit Processes
• GDPP: Learning Diverse Generations Using Determinantal Point Process
• Information Losses in Neural Classifiers from Sampling
• Wasserstein Variational Inference
• Training GANs with Stronger Augmentations via Contrastive Discriminator
• Self-Supervised GANs with Label Augmentation
• Label-Noise Robust Generative Adversarial Networks
• Image Synthesis From Reconfigurable Layout and Style
• Implicit Policy for Reinforcement Learning
• Kernel Implicit Variational Inference
• Copula Variational Bayes inference via information geometry
• Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference
• Temporally coherent video anonymization through GAN inpainting
• Adaptive Weighted Discriminator for Training Generative Adversarial Networks
• Towards Audio to Scene Image Synthesis using Generative Adversarial Network
• Class-Distinct and Class-Mutual Image Generation with GANs
• Filtering Variational Objectives
• Deep Consensus Learning
• Doubly Semi-Implicit Variational Inference
• A Novel Framework for Selection of GANs for an Application
• Stable Rank Normalization for Improved Generalization in Neural Networks and GANs
• Slimmable Generative Adversarial Networks
• Capturing the diversity of biological tuning curves using generative adversarial networks
• Easy High-Dimensional Likelihood-Free Inference
• A Contemporary Overview of Probabilistic Latent Variable Models
• ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models
• Forward Amortized Inference for Likelihood-Free Variational Marginalization
• Latent Transformations for Discrete-Data Normalising Flows
• Neural Embedding for Physical Manipulations