Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling

Abstract

The development of efficient surrogates for partial differential equations (PDEs) is a critical step towards scalable modeling of complex, multiscale systems-of-systems. Convolutional neural networks (CNNs) have gained popularity as the basis for such surrogate models due to their success in capturing high-dimensional input-output mappings and the negligible cost of a forward pass. However, the high cost of generating training data -- typically via classical numerical solvers -- raises the question of whether these models are worth pursuing over more straightforward alternatives with well-established theoretical foundations, such as Monte Carlo methods. To reduce the cost of data generation, we propose training a CNN surrogate model on a mixture of numerical solutions to both the $d$-dimensional problem and its ($d-1$)-dimensional approximation, taking advantage of the efficiency savings guaranteed by the curse of dimensionality. We demonstrate our approach on a multiphase flow test problem, using transfer learning to train a dense fully-convolutional encoder-decoder CNN on the two classes of data. Numerical results from a sample uncertainty quantification task demonstrate that our surrogate model outperforms Monte Carlo with several times the data generation budget.

References in corpus (10)

• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media
• A support vector regression-based multi-fidelity surrogate model
• Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems
• Deep Learning for Simultaneous Inference of Hydraulic and Transport Properties
• Convergence Rates for Learning Linear Operators from Noisy Data
• Overcoming the curse of dimensionality in the numerical approximation of parabolic partial differential equations with gradient-dependent nonlinearities
• A multifidelity approach to continual learning for physical systems