A novel meta-learning initialization method for physics-informed neural networks

Abstract

Physics-informed neural networks (PINNs) have been widely used to solve various scientific computing problems. However, large training costs limit PINNs for some real-time applications. Although some works have been proposed to improve the training efficiency of PINNs, few consider the influence of initialization. To this end, we propose a New Reptile initialization based Physics-Informed Neural Network (NRPINN). The original Reptile algorithm is a meta-learning initialization method based on labeled data. PINNs can be trained with less labeled data or even without any labeled data by adding partial differential equations (PDEs) as a penalty term into the loss function. Inspired by this idea, we propose the new Reptile initialization to sample more tasks from the parameterized PDEs and adapt the penalty term of the loss. The new Reptile initialization can acquire initialization parameters from related tasks by supervised, unsupervised, and semi-supervised learning. Then, PINNs with initialization parameters can efficiently solve PDEs. Besides, the new Reptile initialization can also be used for the variants of PINNs. Finally, we demonstrate and verify the NRPINN considering both forward problems, including solving Poisson, Burgers, and Schrödinger equations, as well as inverse problems, where unknown parameters in the PDEs are estimated. Experimental results show that the NRPINN training is much faster and achieves higher accuracy than PINNs with other initialization methods.

References in corpus (21)

• Adam: A Method for Stochastic Optimization
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• DGM: A deep learning algorithm for solving partial differential equations
• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition
• Physics-informed neural networks for inverse problems in nano-optics and metamaterials
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• On First-Order Meta-Learning Algorithms
• Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport
• Transfer learning based multi-fidelity physics informed deep neural network
• On the Poisson equation and diffusion approximation 3
• Meta-Learning with Implicit Gradients
• A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
• Fundamentals of Traffic Flow
• Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
• Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks
• Accelerating Physics-Informed Neural Network Training with Prior Dictionaries
• Neural networks catching up with finite differences in solving partial differential equations in higher dimensions

Cited by in corpus (5)

• Multi-Objective Loss Balancing for Physics-Informed Deep Learning
• Towards Foundation Model for Chemical Reactor Modeling: Meta-Learning with Physics-Informed Adaptation
• Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities
• Meta-Auto-Decoder: A Meta-Learning Based Reduced Order Model for Solving Parametric Partial Differential Equations
• Compositional Meta-Learning for Mitigating Task Heterogeneity in Physics-Informed Neural Networks