Designing Accurate Emulators for Scientific Processes using Calibration-Driven Deep Models

Abstract

Predictive models that accurately emulate complex scientific processes can achieve exponential speed-ups over numerical simulators or experiments, and at the same time provide surrogates for improving the subsequent analysis. Consequently, there is a recent surge in utilizing modern machine learning (ML) methods, such as deep neural networks, to build data-driven emulators. While the majority of existing efforts has focused on tailoring off-the-shelf ML solutions to better suit the scientific problem at hand, we study an often overlooked, yet important, problem of choosing loss functions to measure the discrepancy between observed data and the predictions from a model. Due to lack of better priors on the expected residual structure, in practice, simple choices such as the mean squared error and the mean absolute error are made. However, the inherent symmetric noise assumption made by these loss functions makes them inappropriate in cases where the data is heterogeneous or when the noise distribution is asymmetric. We propose Learn-by-Calibrating (LbC), a novel deep learning approach based on interval calibration for designing emulators in scientific applications, that are effective even with heterogeneous data and are robust to outliers. Using a large suite of use-cases, we show that LbC provides significant improvements in generalization error over widely-adopted loss function choices, achieves high-quality emulators even in small data regimes and more importantly, recovers the inherent noise structure without any explicit priors.

References in corpus (6)

• Methods for Interpreting and Understanding Deep Neural Networks
• Searching for Exotic Particles in High-Energy Physics with Deep Learning
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks
• Uncertainties in Parameters Estimated with Neural Networks: Application to Strong Gravitational Lensing

Cited by in corpus (7)

• COSMOPOWER: emulating cosmological power spectra for accelerated Bayesian inference from next-generation surveys
• The data-driven future of high energy density physics
• Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers
• Self-learning Emulators and Eigenvector Continuation
• Reduced basis emulation of pairing in finite systems
• A Spatio-Temporal Neural Network Forecasting Approach for Emulation of Firefront Models
• h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective