A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems

Abstract

We propose a new composite neural network (NN) that can be trained based on multi-fidelity data. It is comprised of three NNs, with the first NN trained using the low-fidelity data and coupled to two high-fidelity NNs, one with activation functions and another one without, in order to discover and exploit nonlinear and linear correlations, respectively, between the low-fidelity and the high-fidelity data. We first demonstrate the accuracy of the new multi-fidelity NN for approximating some standard benchmark functions but also a 20-dimensional function. Subsequently, we extend the recently developed physics-informed neural networks (PINNs) to be trained with multi-fidelity data sets (MPINNs). MPINNs contain four fully-connected neural networks, where the first one approximates the low-fidelity data, while the second and third construct the correlation between the low- and high-fidelity data and produce the multi-fidelity approximation, which is then used in the last NN that encodes the partial differential equations (PDEs). Specifically, in the two high-fidelity NNs a relaxation parameter is introduced, which can be optimized to combine the linear and nonlinear sub-networks. By optimizing this parameter, the present model is capable of learning both the linear and complex nonlinear correlations between the low- and high-fidelity data adaptively. By training the MPINNs, we can:(1) obtain the correlation between the low- and high-fidelity data, (2) infer the quantities of interest based on a few scattered data, and (3) identify the unknown parameters in the PDEs. In particular, we employ the MPINNs to learn the hydraulic conductivity field for unsaturated flows as well as the reactive models for reactive transport. The results demonstrate that MPINNs can achieve relatively high accuracy based on a very small set of high-fidelity data.

References in corpus (8)

• Practical implementation of nonlinear time series methods: The TISEAN package
• Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Inferring solutions of differential equations using noisy multi-fidelity data
• Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data
• Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks
• Deep Multi-fidelity Gaussian Processes
• Identification of physical processes via combined data-driven and data-assimilation methods

Cited by in corpus (109)

• DeepXDE: A deep learning library for solving differential equations
• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain
• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• Efficient training of physics-informed neural networks via importance sampling
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• Uncovering near-wall blood flow from sparse data with physics-informed neural networks
• Transfer learning based multi-fidelity physics informed deep neural network
• Learning Properties of Ordered and Disordered Materials from Multi-fidelity Data
• Multi-fidelity Bayesian Neural Networks: Algorithms and Applications
• Non-invasive Inference of Thrombus Material Properties with Physics-informed Neural Networks
• Deep Learning in Deterministic Computational Mechanics
• Recent Advances and Applications of Machine Learning in Experimental Solid Mechanics: A Review
• Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method
• A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic
• Review of multi-fidelity models
• Meta-learning PINN loss functions
• Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities
• Reliable extrapolation of deep neural operators informed by physics or sparse observations
• Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion
• Multi-fidelity surrogate modeling using long short-term memory networks
• Deep Learning for One-dimensional Consolidation
• Explore missing flow dynamics by physics-informed deep learning: the parameterised governing systems
• Physics-informed neural networks (PINNs) for fluid mechanics: A review
• Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems
• On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton--Jacobi partial differential equations
• Inverse design optimization framework via a two-step deep learning approach: application to a wind turbine airfoil
• Poisson CNN: Convolutional neural networks for the solution of the Poisson equation on a Cartesian mesh
• On-the-fly adaptivity for nonlinear twoscale simulations using artificial neural networks and reduced order modeling
• Learning Functional Priors and Posteriors from Data and Physics
• Multi-fidelity prediction of fluid flow and temperature field based on transfer learning using Fourier Neural Operator
• Use of Multifidelity Training Data and Transfer Learning for Efficient Construction of Subsurface Flow Surrogate Models
• Combining physics-based and data-driven models: advancing the frontiers of research with Scientific Machine Learning
• Extraction of Material Properties through Multi-fidelity Deep Learning from Molecular Dynamics Simulation
• Active- and transfer-learning applied to microscale-macroscale coupling to simulate viscoelastic flows
• Hybrid Machine Learning for Scanning Near-field Optical Spectroscopy
• Artificial intelligence approaches for materials-by-design of energetic materials: state-of-the-art, challenges, and future directions
• GINNs: Graph-Informed Neural Networks for Multiscale Physics
• A fast multi-fidelity method with uncertainty quantification for complex data correlations: Application to vortex-induced vibrations of marine risers
• Dynamic fracture of a bicontinuously nanostructured copolymer: A deep-learning analysis of big-data-generating experiment
• Machine learning spectral indicators of topology
• Deep neural network enabled corrective source term approach to hybrid analysis and modeling
• A DeepONet multi-fidelity approach for residual learning in reduced order modeling
• Neural Network Training Using ${\ell }_1$-Regularization and Bi-fidelity Data
• A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networks
• CEENs: Causality-enforced evolutional networks for solving time-dependent partial differential equations
• A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
• Deep encoder-decoder hierarchical convolutional neural networks for conjugate heat transfer surrogate modeling
• Physics-integrated hybrid framework for model form error identification in nonlinear dynamical systems
• Solving time dependent Fokker-Planck equations via temporal normalizing flow
• Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks
• Physics-informed neural networks for unsteady incompressible flows with time-dependent moving boundaries
• Surrogate modelling and uncertainty quantification based on multi-fidelity deep neural network
• Physics-informed neural networks for understanding shear migration of particles in viscous flow
• Physics Constrained Learning for Data-driven Inverse Modeling from Sparse Observations
• Multi-fidelity Bayesian Optimization: A Review
• Neural network architectures using min-plus algebra for solving certain high dimensional optimal control problems and Hamilton-Jacobi PDEs
• Probabilistic Neural Data Fusion for Learning from an Arbitrary Number of Multi-fidelity Data Sets
• Equivariant geometric learning for digital rock physics: estimating formation factor and effective permeability tensors from Morse graph
• Physics-informed neural networks for solving forward and inverse flow problems via the Boltzmann-BGK formulation
• Physics-informed deep learning for flow and deformation in poroelastic media
• Mutual Information for Explainable Deep Learning of Multiscale Systems
• Multi-fidelity data fusion through parameter space reduction with applications to automotive engineering
• Improved Surrogate Modeling of Fluid Dynamics with Physics-Informed Neural Networks
• A multifidelity approach to continual learning for physical systems
• A Data-driven Multi-fidelity Physics-informed Learning Framework for Smart Manufacturing: A Composites Processing Case Study
• A Practitioner's Guide to Kolmogorov-Arnold Networks
• GoRINNs: Godunov-Riemann Informed Neural Networks for Learning Hyperbolic Conservation Laws
• Enhancing Bayesian model updating in structural health monitoring via learnable mappings
• Learning and Meta-Learning of Stochastic Advection-Diffusion-Reaction Systems from Sparse Measurements
• An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems
• Residual Multi-Fidelity Neural Network Computing
• Bridging Fidelities to Predict Nanoindentation Tip Radii Using Interpretable Deep Learning Models
• A Bayesian neural network approach to Multi-fidelity surrogate modelling
• Splitting physics-informed neural networks for inferring the dynamics of integer- and fractional-order neuron models
• Transfer Learning of High-Fidelity Opacity Spectra in Autoencoders and Surrogate Models
• Transfer Learning as a Method to Reproduce High-Fidelity NLTE Opacities in Simulations
• Overcoming the curse of dimensionality for some Hamilton--Jacobi partial differential equations via neural network architectures
• Multi-fidelity Hierarchical Neural Processes
• Multi-fidelity data fusion for the approximation of scalar functions with low intrinsic dimensionality through active subspaces
• Flow Matching for Accelerated Simulation of Atomic Transport in Crystalline Materials
• How machine learning can help the design and analysis of composite materials and structures?
• Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling
• Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling
• Constructing Sub-scale Surrogate Model for Proppant Settling in Inclined Fractures from Simulation Data with Multi-fidelity Neural Network
• Physics-informed neural networks for improving cerebral hemodynamics predictions
• GAN-MDF: A Method for Multi-fidelity Data Fusion in Digital Twins
• Assessing Non-Nested Configurations of Multifidelity Machine Learning for Quantum-Chemical Properties
• Surrogate Modeling of Landau Damping with Deep Operator Networks
• Parameter identification for a damage model using a physics informed neural network
• A transfer learning metamodel using artificial neural networks applied to natural convection flows in enclosures
• Solving physics-based initial value problems with unsupervised machine learning
• Uncertainty Quantification of Locally Nonlinear Dynamical Systems using Neural Networks
• Graph Laplacian-based Bayesian Multi-fidelity Modeling
• A hierarchical neural hybrid method for failure probability estimation
• Physics-informed neural networks viewpoint for solving the Dyson-Schwinger equations of quantum electrodynamics
• A New Artificial Neuron Proposal with Trainable Simultaneous Local and Global Activation Function
• Neural network-based Godunov corrections for approximate Riemann solvers using bi-fidelity learning
• Adaptive Computing for Scale-up Problems
• On the performance of multi-fidelity and reduced-dimensional neural emulators for inference of physiological boundary conditions
• Efficient Characterization of Dynamic Response Variation Using Multi-Fidelity Data Fusion through Composite Neural Network
• An acceleration strategy for randomize-then-optimize sampling via deep neural networks
• Extending deep learning U-Net architecture for predicting unsteady fluid flows in textured microchannels
• Rational function approximation with normalized positive denominators
• MFPC-Net: Multi-fidelity Physics-Constrained Neural Process
• Projection-based multifidelity linear regression for data-scarce applications