Hierarchical Variational Models

Abstract

Black box variational inference allows researchers to easily prototype and evaluate an array of models. Recent advances allow such algorithms to scale to high dimensions. However, a central question remains: How to specify an expressive variational distribution that maintains efficient computation? To address this, we develop hierarchical variational models (HVMs). HVMs augment a variational approximation with a prior on its parameters, which allows it to capture complex structure for both discrete and continuous latent variables. The algorithm we develop is black box, can be used for any HVM, and has the same computational efficiency as the original approximation. We study HVMs on a variety of deep discrete latent variable models. HVMs generalize other expressive variational distributions and maintains higher fidelity to the posterior.

Appears in International Conference on Machine Learning, 2016

References in corpus (9)

• Variational Inference with Normalizing Flows
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Auxiliary Deep Generative Models
• Edward: A library for probabilistic modeling, inference, and criticism
• Automatic Differentiation Variational Inference
• Automatic Variational Inference in Stan
• Structured Stochastic Variational Inference
• Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayes
• Local Expectation Gradients for Doubly Stochastic Variational Inference

Cited by in corpus (10)

• Auxiliary Deep Generative Models
• Automatic Differentiation Variational Inference
• Improving Variational Inference with Inverse Autoregressive Flow
• Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
• Fixing a Broken ELBO
• Truncated Variational Expectation Maximization
• Bottleneck Conditional Density Estimation
• Note on the equivalence of hierarchical variational models and auxiliary deep generative models
• Adversarial Variational Bayes Methods for Tweedie Compound Poisson Mixed Models
• ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models