Calibrating Over-Parametrized Simulation Models: A Framework via Eligibility Set

Abstract

Stochastic simulation aims to compute output performance for complex models that lack analytical tractability. To ensure accurate prediction, the model needs to be calibrated and validated against real data. Conventional methods approach these tasks by assessing the model-data match via simple hypothesis tests or distance minimization in an ad hoc fashion, but they can encounter challenges arising from non-identifiability and high dimensionality. In this paper, we investigate a framework to develop calibration schemes that satisfy rigorous frequentist statistical guarantees, via a basic notion that we call eligibility set designed to bypass non-identifiability via a set-based estimation. We investigate a feature extraction-then-aggregation approach to construct these sets that target at multivariate outputs. We demonstrate our methodology on several numerical examples, including an application to calibration of a limit order book market simulator (ABIDES).

References in corpus (6)

• The frontier of simulation-based inference
• Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques
• MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification
• Image-based model parameter optimization using Model-Assisted Generative Adversarial Networks
• Efficient Calibration of Multi-Agent Simulation Models from Output Series with Bayesian Optimization
• A theoretical framework for calibration in computer models: parametrization, estimation and convergence properties