L-HYDRA: Multi-Head Physics-Informed Neural Networks

Abstract

We introduce multi-head neural networks (MH-NNs) to physics-informed machine learning, which is a type of neural networks (NNs) with all nonlinear hidden layers as the body and multiple linear output layers as multi-head. Hence, we construct multi-head physics-informed neural networks (MH-PINNs) as a potent tool for multi-task learning (MTL), generative modeling, and few-shot learning for diverse problems in scientific machine learning (SciML). MH-PINNs connect multiple functions/tasks via a shared body as the basis functions as well as a shared distribution for the head. The former is accomplished by solving multiple tasks with MH-PINNs with each head independently corresponding to each task, while the latter by employing normalizing flows (NFs) for density estimate and generative modeling. To this end, our method is a two-stage method, and both stages can be tackled with standard deep learning tools of NNs, enabling easy implementation in practice. MH-PINNs can be used for various purposes, such as approximating stochastic processes, solving multiple tasks synergistically, providing informative prior knowledge for downstream few-shot learning tasks such as meta-learning and transfer learning, learning representative basis functions, and uncertainty quantification. We demonstrate the effectiveness of MH-PINNs in five benchmarks, investigating also the possibility of synergistic learning in regression analysis. We name the open-source code "Lernaean Hydra" (L-HYDRA), since this mythical creature possessed many heads for performing important multiple tasks, as in the proposed method.

References in corpus (4)

• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems
• PI-VAE: Physics-Informed Variational Auto-Encoder for stochastic differential equations
• Multi-Output Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Uncertainties

Cited by in corpus (2)

• Uncertainty quantification and stability of neural operators for prediction of three-dimensional turbulence
• Fast meta-solvers for 3D complex-shape scatterers using neural operators trained on a non-scattering problem