Probabilistically-autoencoded horseshoe-disentangled multidomain item-response theory models

Abstract

Item response theory (IRT) is a non-linear generative probabilistic paradigm for using exams to identify, quantify, and compare latent traits of individuals, relative to their peers, within a population of interest. In pre-existing multidimensional IRT methods, one requires a factorization of the test items. For this task, linear exploratory factor analysis is used, making IRT a posthoc model. We propose skipping the initial factor analysis by using a sparsity-promoting horseshoe prior to perform factorization directly within the IRT model so that all training occurs in a single self-consistent step. Being a hierarchical Bayesian model, we adapt the WAIC to the problem of dimensionality selection. IRT models are analogous to probabilistic autoencoders. By binding the generative IRT model to a Bayesian neural network (forming a probabilistic autoencoder), one obtains a scoring algorithm consistent with the interpretable Bayesian model. In some IRT applications the black-box nature of a neural network scoring machine is desirable. In this manuscript, we demonstrate within-IRT factorization and comment on scoring approaches.

arXiv commentPresented as poster at the NeurIPS 2019 Bayesian Deep Learning workshop

References in corpus (12)

• Auto-Encoding Variational Bayes
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory
• Weight Uncertainty in Neural Networks
• Sparsity information and regularization in the horseshoe and other shrinkage priors
• Comparison of Bayesian predictive methods for model selection
• A simple sampler for the horseshoe estimator
• Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches
• Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors
• Predictive Bayesian selection of multistep Markov chains, applied to the detection of the hot hand and other statistical dependencies in free throws
• Disentangling and Learning Robust Representations with Natural Clustering
• On the half-Cauchy prior for a global scale parameter

Cited by in corpus (2)

• Sparse encoding for more-interpretable feature-selecting representations in probabilistic matrix factorization
• Regularized Bayesian calibration and scoring of the WD-FAB IRT model improves predictive performance over marginal maximum likelihood