Variational inference for cutting feedback in misspecified models
arXiv:2108.11066
Abstract
Bayesian analyses combine information represented by different terms in a joint Bayesian model. When one or more of the terms is misspecified, it can be helpful to restrict the use of information from suspect model components to modify posterior inference. This is called "cutting feedback", and both the specification and computation of the posterior for such "cut models" is challenging. In this paper, we define cut posterior distributions as solutions to constrained optimization problems, and propose optimization-based variational methods for their computation. These methods are faster than existing Markov chain Monte Carlo (MCMC) approaches for computing cut posterior distributions by an order of magnitude. It is also shown that variational methods allow for the evaluation of computationally intensive conflict checks that can be used to decide whether or not feedback should be cut. Our methods are illustrated in a number of simulated and real examples, including an application where recent methodological advances that combine variational inference and MCMC within the variational optimization are used.
References in corpus (6)
- ADADELTA: An Adaptive Learning Rate Method
- Generalized Variational Inference: Three arguments for deriving new Posteriors
- Better together? Statistical learning in models made of modules
- Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations
- Stochastic Approximation Cut Algorithm for Inference in Modularized Bayesian Models
- Semi-Modular Inference: enhanced learning in multi-modular models by tempering the influence of components