Residual Multi-Fidelity Neural Network Computing

Abstract

In this work, we consider the general problem of constructing a neural network surrogate model using multi-fidelity information. Motivated by error-complexity estimates for ReLU neural networks, we formulate the correlation between an inexpensive low-fidelity model and an expensive high-fidelity model as a possibly non-linear residual function. This function defines a mapping between 1) the shared input space of the models along with the low-fidelity model output, and 2) the discrepancy between the outputs of the two models. The computational framework proceeds by training two neural networks to work in concert. The first network learns the residual function on a small set of high- and low-fidelity data. Once trained, this network is used to generate additional synthetic high-fidelity data, which is used in the training of the second network. The trained second network then acts as our surrogate for the high-fidelity quantity of interest. We present four numerical examples to demonstrate the power of the proposed framework, showing that significant savings in computational cost may be achieved when the output predictions are desired to be accurate within small tolerances.

References in corpus (8)

• Deep Learning in Neural Networks: An Overview
• DGM: A deep learning algorithm for solving partial differential equations
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
• Multi-fidelity Bayesian Neural Networks: Algorithms and Applications
• Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities
• Multi-fidelity surrogate modeling using long short-term memory networks
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems