Nonparametric variational inference

Abstract

Variational methods are widely used for approximate posterior inference. However, their use is typically limited to families of distributions that enjoy particular conjugacy properties. To circumvent this limitation, we propose a family of variational approximations inspired by nonparametric kernel density estimation. The locations of these kernels and their bandwidth are treated as variational parameters and optimized to improve an approximate lower bound on the marginal likelihood of the data. Using multiple kernels allows the approximation to capture multiple modes of the posterior, unlike most other variational approximations. We demonstrate the efficacy of the nonparametric approximation with a hierarchical logistic regression model and a nonlinear matrix factorization model. We obtain predictive performance as good as or better than more specialized variational methods and sample-based approximations. The method is easy to apply to more general graphical models for which standard variational methods are difficult to derive.

ICML2012

References in corpus (2)

• MCMC using Hamiltonian dynamics
• Variational inference for large-scale models of discrete choice

Cited by in corpus (19)

• Variational Inference with Normalizing Flows
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
• Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors
• Stochastic Variational Inference
• Advances in Variational Inference
• Particle-based Energetic Variational Inference
• Infinite-dimensional gradient-based descent for alpha-divergence minimisation
• Embarrassingly Parallel Variational Inference in Nonconjugate Models
• Kernel Stein Discrepancy Descent
• Ensemble Model Patching: A Parameter-Efficient Variational Bayesian Neural Network
• Trust-Region Variational Inference with Gaussian Mixture Models
• Interpolating between sampling and variational inference with infinite stochastic mixtures
• Adaptive Importance Sampling meets Mirror Descent: a Bias-variance tradeoff
• Visual Attention in Imaginative Agents
• Copula-like Variational Inference
• BooVAE: Boosting Approach for Continual Learning of VAE
• Manifold Optimization Assisted Gaussian Variational Approximation
• Particle Flow Bayes' Rule
• Mixed Variational Inference