Variational Boosting: Iteratively Refining Posterior Approximations
arXiv:1611.06585
Abstract
We propose a black-box variational inference method to approximate intractable distributions with an increasingly rich approximating class. Our method, termed variational boosting, iteratively refines an existing variational approximation by solving a sequence of optimization problems, allowing the practitioner to trade computation time for accuracy. We show how to expand the variational approximating class by incorporating additional covariance structure and by introducing new components to form a mixture. We apply variational boosting to synthetic and real statistical models, and show that resulting posterior inferences compare favorably to existing posterior approximation algorithms in both accuracy and efficiency.
25 pages, 9 figures, 2 tables
Cited by in corpus (21)
- Quality of Uncertainty Quantification for Bayesian Neural Network Inference
- Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors
- Variational Bayesian Monte Carlo with Noisy Likelihoods
- Advances in Variational Inference
- The k-tied Normal Distribution: A Compact Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural Networks
- Gaussian variational approximation for high-dimensional state space models
- Validated Variational Inference via Practical Posterior Error Bounds
- Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergence
- Recursive Inference for Variational Autoencoders
- Ensemble Model Patching: A Parameter-Efficient Variational Bayesian Neural Network
- Challenges and Opportunities in High-dimensional Variational Inference
- Variational Inference with Holder Bounds
- Interpolating between sampling and variational inference with infinite stochastic mixtures
- Variational Marginal Particle Filters
- Copula-like Variational Inference
- Variational Inference with Mixture Model Approximation: Robotic Applications
- Pathwise Derivatives for Multivariate Distributions
- Reliable Categorical Variational Inference with Mixture of Discrete Normalizing Flows
- BooVAE: Boosting Approach for Continual Learning of VAE
- Gradient Boosted Normalizing Flows
- Riemannian Laplace approximations for Bayesian neural networks