GPdoemd: a Python package for design of experiments for model discrimination

Abstract

Model discrimination identifies a mathematical model that usefully explains and predicts a given system's behaviour. Researchers will often have several models, i.e. hypotheses, about an underlying system mechanism, but insufficient experimental data to discriminate between the models, i.e. discard inaccurate models. Given rival mathematical models and an initial experimental data set, optimal design of experiments suggests maximally informative experimental observations that maximise a design criterion weighted by prediction uncertainty. The model uncertainty requires gradients, which may not be readily available for black-box models. This paper (i) proposes a new design criterion using the Jensen-Rényi divergence, and (ii) develops a novel method replacing black-box models with Gaussian process surrogates. Using the surrogates, we marginalise out the model parameters with approximate inference. Results show these contributions working well for both classical and new test instances. We also (iii) introduce and discuss GPdoemd, the open-source implementation of the Gaussian process surrogate method.

References in corpus (7)

• Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory
• A Widely Applicable Bayesian Information Criterion
• Automated derivation of the adjoint of high-level transient finite element programs
• Batch Bayesian Optimization via Local Penalization
• When Gaussian Process Meets Big Data: A Review of Scalable GPs
• Variational Fourier features for Gaussian processes
• Scalable Gaussian Processes with Grid-Structured Eigenfunctions (GP-GRIEF)

Cited by in corpus (3)

• Simplified algorithms for adaptive experiment design in parameter estimation
• Robust multi-stage model-based design of optimal experiments for nonlinear estimation
• Using Gaussian Processes to Design Dynamic Experiments for Black-Box Model Discrimination under Uncertainty