Bayesian comparison of latent variable models: Conditional vs marginal likelihoods

Abstract

Typical Bayesian methods for models with latent variables (or random effects) involve directly sampling the latent variables along with the model parameters. In high-level software code for model definitions (using, e.g., BUGS, JAGS, Stan), the likelihood is therefore specified as conditional on the latent variables. This can lead researchers to perform model comparisons via conditional likelihoods, where the latent variables are considered model parameters. In other settings, however, typical model comparisons involve marginal likelihoods where the latent variables are integrated out. This distinction is often overlooked despite the fact that it can have a large impact on the comparisons of interest. In this paper, we clarify and illustrate these issues, focusing on the comparison of conditional and marginal Deviance Information Criteria (DICs) and Watanabe-Akaike Information Criteria (WAICs) in psychometric modeling. The conditional/marginal distinction corresponds to whether the model should be predictive for the clusters that are in the data or for new clusters (where "clusters" typically correspond to higher-level units like people or schools). Correspondingly, we show that marginal WAIC corresponds to leave-one-cluster out (LOcO) cross-validation, whereas conditional WAIC corresponds to leave-one-unit out (LOuO). These results lead to recommendations on the general application of the criteria to models with latent variables.

arXiv commentManuscript in press at Psychometrika; 31 pages, 8 figures

References in corpus (9)

• Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
• Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory
• A weakly informative default prior distribution for logistic and other regression models
• Using stacking to average Bayesian predictive distributions
• Comparison of Bayesian predictive methods for model selection
• blavaan: Bayesian structural equation models via parameter expansion
• Bayesian leave-one-out cross-validation approximations for Gaussian latent variable models
• Approximating Cross-validatory Predictive Evaluation in Bayesian Latent Variables Models with Integrated IS and WAIC
• Limitations of "Limitations of Bayesian leave-one-out cross-validation for model selection"

Cited by in corpus (11)

• Efficient Bayesian Structural Equation Modeling in Stan
• Computation and application of generalized linear mixed model derivatives using lme4
• A guide to state-space modeling of ecological time series
• Opaque prior distributions in Bayesian latent variable models
• Uncertainty in Bayesian Leave-One-Out Cross-Validation Based Model Comparison
• Efficient estimation and correction of selection-induced bias with order statistics
• Approximate Cross-validated Mean Estimates for Bayesian Hierarchical Regression Models
• A Comparison between Markov Switching Zero-inflated and Hurdle Models for Spatio-temporal Infectious Disease Counts
• Gaussian distributional structural equation models: A framework for modeling latent heteroscedasticity
• A Continuous-Time Dynamic Factor Model for Intensive Longitudinal Data Arising from Mobile Health Studies
• A Restricted Latent Class Hidden Markov Model for Polytomous Responses, Polytomous Attributes, and Covariates: Identifiability and Application