Meta-learning PINN loss functions

Abstract

We propose a meta-learning technique for offline discovery of physics-informed neural network (PINN) loss functions. We extend earlier works on meta-learning, and develop a gradient-based meta-learning algorithm for addressing diverse task distributions based on parametrized partial differential equations (PDEs) that are solved with PINNs. Furthermore, based on new theory we identify two desirable properties of meta-learned losses in PINN problems, which we enforce by proposing a new regularization method or using a specific parametrization of the loss function. In the computational examples, the meta-learned losses are employed at test time for addressing regression and PDE task distributions. Our results indicate that significant performance improvement can be achieved by using a shared-among-tasks offline-learned loss function even for out-of-distribution meta-testing. In this case, we solve for test tasks that do not belong to the task distribution used in meta-training, and we also employ PINN architectures that are different from the PINN architecture used in meta-training. To better understand the capabilities and limitations of the proposed method, we consider various parametrizations of the loss function and describe different algorithm design options and how they may affect meta-learning performance.

References in corpus (9)

• Understanding and mitigating gradient pathologies in physics-informed neural networks
• Learning to Learn: Meta-Critic Networks for Sample Efficient Learning
• Generalized Inner Loop Meta-Learning
• Improving Generalization in Meta Reinforcement Learning using Learned Objectives
• Reward Shaping via Meta-Learning
• Learning Surrogate Losses
• Addressing the Loss-Metric Mismatch with Adaptive Loss Alignment
• A Recipe for Global Convergence Guarantee in Deep Neural Networks
• Effective Regularization Through Loss-Function Metalearning

Cited by in corpus (14)

• A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
• Super-Resolution Analysis via Machine Learning: A Survey for Fluid Flows
• Physics-informed neural network methods based on Miura transformations and discovery of new localized wave solutions
• Physics-Informed Deep Learning For Traffic State Estimation: A Survey and the Outlook
• HANNA: Hard-constraint Neural Network for Consistent Activity Coefficient Prediction
• Hutchinson Trace Estimation for High-Dimensional and High-Order Physics-Informed Neural Networks
• Learning Specialized Activation Functions for Physics-informed Neural Networks
• PINN Training using Biobjective Optimization: The Trade-off between Data Loss and Residual Loss
• Accelerating physics-informed neural network based 1D arc simulation by meta learning
• Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings
• Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities
• Meta-Auto-Decoder: A Meta-Learning Based Reduced Order Model for Solving Parametric Partial Differential Equations
• Neural Networks with Inputs Based on Domain of Dependence and A Converging Sequence for Solving Conservation Laws, Part I: 1D Riemann Problems
• Gated X-TFC: Soft Domain Decomposition for Forward and Inverse Problems in Sharp-Gradient PDEs