Accuracy of Gaussian approximation in nonparametric Bernstein -- von Mises Theorem
arXiv:1910.06028
Abstract
The prominent Bernstein -- von Mises (BvM) result claims that the posterior distribution after centering by the efficient estimator and standardizing by the square root of the total Fisher information is nearly standard normal. In particular, the prior completely washes out from the asymptotic posterior distribution. This fact is fundamental and justifies the Bayes approach from the frequentist viewpoint. In the nonparametric setup the situation changes dramatically and the impact of prior becomes essential even for the contraction of the posterior; see [vdV2008], [Bo2011], [CaNi2013,CaNi2014] for different models like Gaussian regression or i.i.d. model in different weak topologies. This paper offers another non-asymptotic approach to studying the behavior of the posterior for a special but rather popular and useful class of statistical models and for Gaussian priors. First we derive tight finite sample bounds on posterior contraction in terms of the so called effective dimension of the parameter space. Our main results describe the accuracy of Gaussian approximation of the posterior. In particular, we show that restricting to the class of all centrally symmetric credible sets around pMLE allows to get Gaussian approximation up to order (n^{-1}). We also show that the posterior distribution mimics well the distribution of the penalized maximum likelihood estimator (pMLE) and reduce the question of reliability of credible sets to consistency of the pMLE-based confidence sets. The obtained results are specified for nonparametric log-density estimation and generalized regression.
References in corpus (7)
- Rates of contraction of posterior distributions based on Gaussian process priors
- On the Bernstein-von Mises phenomenon for nonparametric Bayes procedures
- Parametric estimation. Finite sample theory
- Bernstein von Mises Theorems for Gaussian Regression with increasing number of regressors
- Finite Sample Bernstein -- von Mises Theorem for Semiparametric Problems
- Large ball probability, Gaussian comparison and anti-concentration
- Bayesian inference for nonlinear inverse problems