Brittleness of Bayesian Inference Under Finite Information in a Continuous World
arXiv:1304.6772 · doi:10.1214/15-EJS989
Abstract
We derive, in the classical framework of Bayesian sensitivity analysis, optimal lower and upper bounds on posterior values obtained from Bayesian models that exactly capture an arbitrarily large number of finite-dimensional marginals of the data-generating distribution and/or that are as close as desired to the data-generating distribution in the Prokhorov or total variation metrics; these bounds show that such models may still make the largest possible prediction error after conditioning on an arbitrarily large number of sample data measured at finite precision. These results are obtained through the development of a reduction calculus for optimization problems over measures on spaces of measures. We use this calculus to investigate the mechanisms that generate brittleness/robustness and, in particular, we observe that learning and robustness are antagonistic properties. It is now well understood that the numerical resolution of PDEs requires the satisfaction of specific stability conditions. Is there a missing stability condition for using Bayesian inference in a continuous world under finite information?
arXiv comment81 pages. To appear in Electronic Journal of Statistics
References in corpus (5)
Cited by in corpus (12)
- Deep Learning of Vortex Induced Vibrations
- Kernel Flows: from learning kernels from data into the abyss
- Learning dynamical systems from data: a simple cross-validation perspective
- Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients
- On the Brittleness of Bayesian Inference
- A Modern Retrospective on Probabilistic Numerics
- On the Local Lipschitz Stability of Bayesian Inverse Problems
- Random forward models and log-likelihoods in Bayesian inverse problems
- Towards Machine Wald
- Hamiltonian Dynamics of Preferential Attachment
- Bayesian non-parametric inference for -coalescents: consistency and a parametric method
- Generalized Bayes Approach to Inverse Problems with Model Misspecification