Detection of Cognitive Diagnostic Model Misspecification using New Lancaster-Chesher Information Matrix Tests

Abstract

Model specification tests play a crucial role in evaluating the appropriateness of probability models for estimation and inference. Existing methods for the detection of model misspecification such as the chi-square goodness-of-fit (GOF) tests and more recently the M2 statistic \autocite{MaydeuOlivares2005,MaydeuJoe2014} tend to result in test statistics with excessive degrees of freedom for models with larger numbers of parameters. An alternative approach is based upon the Information Matrix (IM) equality. The IM equality asserts that if a probability model is correctly specified, the asymptotic covariance matrix of the maximum likelihood estimators can be asymptotically estimated using a methodology based upon either the first or second derivatives of the log-likelihood function. Using a contrapositive argument, White (1982) \nocite{Wh82} proposed a misspecification test methodology based upon comparing these two alternative covariance matrix estimators. Extending this work, Presnell and Boos (2004) \nocite{Presnell2004} showed how to develop a misspecification test which only requires one degree of freedom regardless of the complexity of the model or data. In this paper, we extend prior work and additionally apply methods of Golden et al. (2013, 2016) \nocite{golden2013}\nocite{Golden2016} to derive and evaluate misspecification tests for Cognitive Diagnostic Models (CDMs) which only require 1 or 2 degrees of freedom regardless of model or data complexity. Analytic formulas for the tests are derived so they can be applied without requiring computationally intensive bootstrap simulation methods. Our simulation studies show the asymptotic statistical tests have good level (type 1 error) and power performance for CDM models and data which might be encountered in practice.