Conflict Diagnostics in Directed Acyclic Graphs, with Applications in Bayesian Evidence Synthesis

Abstract

Complex stochastic models represented by directed acyclic graphs (DAGs) are increasingly employed to synthesise multiple, imperfect and disparate sources of evidence, to estimate quantities that are difficult to measure directly. The various data sources are dependent on shared parameters and hence have the potential to conflict with each other, as well as with the model. In a Bayesian framework, the model consists of three components: the prior distribution, the assumed form of the likelihood and structural assumptions. Any of these components may be incompatible with the observed data. The detection and quantification of such conflict and of data sources that are inconsistent with each other is therefore a crucial component of the model criticism process. We first review Bayesian model criticism, with a focus on conflict detection, before describing a general diagnostic for detecting and quantifying conflict between the evidence in different partitions of a DAG. The diagnostic is a p-value based on splitting the information contributing to inference about a "separator" node or group of nodes into two independent groups and testing whether the two groups result in the same inference about the separator node(s). We illustrate the method with three comprehensive examples: an evidence synthesis to estimate HIV prevalence; an evidence synthesis to estimate influenza case-severity; and a hierarchical growth model for rat weights.

Published in at http://dx.doi.org/10.1214/13-STS426 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (4)

• Relaxation Penalties and Priors for Plausible Modeling of Nonidentified Bias Sources
• Bayesian Checking of the Second Levels of Hierarchical Models
• Weak Informativity and the Information in One Prior Relative to Another
• Invariant $P$-values for model checking

Cited by in corpus (7)

• Joining and splitting models with Markov melding
• A stochastic variational framework for fitting and diagnosing generalized linear mixed models
• Using prior expansions for prior-data conflict checking
• Holdout predictive checks for Bayesian model criticism
• Fast and accurate Bayesian model criticism and conflict diagnostics using R-INLA
• Bayesian multistate modelling of incomplete chronic disease burden data
• Synthesising evidence to estimate pandemic (2009) A/H1N1 influenza severity in 2009-2011