Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs
arXiv:1908.09744
Abstract
A framework to boost the efficiency of Bayesian inference in probabilistic programs is introduced by embedding a sampler inside a variational posterior approximation. We call it the refined variational approximation. Its strength lies both in ease of implementation and automatically tuning of the sampler parameters to speed up mixing time using automatic differentiation. Several strategies to approximate \emph{evidence lower bound} (ELBO) computation are introduced. Experimental evidence of its efficient performance is shown solving an influence diagram in a high-dimensional space using a conditional variational autoencoder (cVAE) as a deep Bayes classifier; an unconditional VAE on density estimation tasks; and state-space models for time-series data.
References in corpus (22)
- Adam: A Method for Stochastic Optimization
- Variational Inference with Normalizing Flows
- Pyro: Deep Universal Probabilistic Programming
- Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
- A General Framework for the Parametrization of Hierarchical Models
- Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks
- Neural Network Renormalization Group
- Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
- Inference Suboptimality in Variational Autoencoders
- Variational Inference using Implicit Distributions
- NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport
- Forward and Reverse Gradient-Based Hyperparameter Optimization
- Zero-Shot Learning via Class-Conditioned Deep Generative Models
- A Spectral Approach to Gradient Estimation for Implicit Distributions
- Simple, Distributed, and Accelerated Probabilistic Programming
- Learning to Draw Samples with Amortized Stein Variational Gradient Descent
- Continuous-Time Flows for Efficient Inference and Density Estimation
- Semi-Implicit Variational Inference
- Stochastic Gradient MCMC with Repulsive Forces
- A Contrastive Divergence for Combining Variational Inference and MCMC
- Unbiased Implicit Variational Inference
- Implicit Deep Latent Variable Models for Text Generation