Validated Variational Inference via Practical Posterior Error Bounds
arXiv:1910.04102
Abstract
Variational inference has become an increasingly attractive fast alternative to Markov chain Monte Carlo methods for approximate Bayesian inference. However, a major obstacle to the widespread use of variational methods is the lack of post-hoc accuracy measures that are both theoretically justified and computationally efficient. In this paper, we provide rigorous bounds on the error of posterior mean and uncertainty estimates that arise from full-distribution approximations, as in variational inference. Our bounds are widely applicable, as they require only that the approximating and exact posteriors have polynomial moments. Our bounds are also computationally efficient for variational inference because they require only standard values from variational objectives, straightforward analytic calculations, and simple Monte Carlo estimates. We show that our analysis naturally leads to a new and improved workflow for validated variational inference. Finally, we demonstrate the utility of our proposed workflow and error bounds on a robust regression problem and on a real-data example with a widely used multilevel hierarchical model.
A python package for carrying out our validated variational inference workflow -- including doing black-box variational inference and computing the bounds we develop in this paper -- is available at https://github.com/jhuggins/viabel. The same repository also contains code for reproducing all of our experiments
References in corpus (18)
- Auto-Encoding Variational Bayes
- Fast Computation of Wasserstein Barycenters
- Pareto Smoothed Importance Sampling
- Frequentist Consistency of Variational Bayes
- Measuring Sample Quality with Stein's Method
- Yes, but Did It Work?: Evaluating Variational Inference
- A characterization of dimension free concentration in terms of transportation inequalities
- Rényi Divergence Variational Inference
- Variational Boosting: Iteratively Refining Posterior Approximations
- Quantitative bounds for Markov chain convergence: Wasserstein and total variation distances
- Boosting Variational Inference
- Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixtures
- (Non-) asymptotic properties of Stochastic Gradient Langevin Dynamics
- Random Feature Stein Discrepancies
- Boosting Variational Inference: an Optimization Perspective
- Universal Boosting Variational Inference
- Boosting Black Box Variational Inference
- Computing the Kullback-Leibler Divergence between two Weibull Distributions