Frequentist Consistency of Generalized Variational Inference
arXiv:1912.04946
Abstract
This paper investigates Frequentist consistency properties of the posterior distributions constructed via Generalized Variational Inference (GVI). A number of generic and novel strategies are given for proving consistency, relying on the theory of -convergence. Specifically, this paper shows that under minimal regularity conditions, the sequence of GVI posteriors is consistent and collapses to a point mass at the population-optimal parameter value as the number of observations goes to infinity. The results extend to the latent variable case without additional assumptions and hold under misspecification. Lastly, the paper explains how to apply the results to a selection of GVI posteriors with especially popular variational families. For example, consistency is established for GVI methods using the mean field normal variational family, normal mixtures, Gaussian process variational families as well as neural networks indexing a normal (mixture) distribution.
arXiv comment31 pages, 3 Figures
References in corpus (14)
- An MCMC Approach to Classical Estimation
- Bayesian inverse problems with Gaussian priors
- Frequentist Consistency of Variational Bayes
- MAP Estimators and Their Consistency in Bayesian Nonparametric Inverse Problems
- Misspecification in infinite-dimensional Bayesian statistics
- New approaches to Bayesian consistency
- Principles of Bayesian Inference using General Divergence Criteria
- Minimum scoring rule inference
- Generalized Variational Inference: Three arguments for deriving new Posteriors
- Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixtures
- Variational Inference based on Robust Divergences
- Asymptotic normality, concentration, and coverage of generalized posteriors
- Bounding errors of Expectation-Propagation
- On the Bernstein-Von Mises Theorem for High Dimensional Nonlinear Bayesian Inverse Problems