When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?

Abstract

Physics-informed neural networks (PINNs) have become a popular choice for solving high-dimensional partial differential equations (PDEs) due to their excellent approximation power and generalization ability. Recently, Extended PINNs (XPINNs) based on domain decomposition methods have attracted considerable attention due to their effectiveness in modeling multiscale and multiphysics problems and their parallelization. However, theoretical understanding on their convergence and generalization properties remains unexplored. In this study, we take an initial step towards understanding how and when XPINNs outperform PINNs. Specifically, for general multi-layer PINNs and XPINNs, we first provide a prior generalization bound via the complexity of the target functions in the PDE problem, and a posterior generalization bound via the posterior matrix norms of the networks after optimization. Moreover, based on our bounds, we analyze the conditions under which XPINNs improve generalization. Concretely, our theory shows that the key building block of XPINN, namely the domain decomposition, introduces a tradeoff for generalization. On the one hand, XPINNs decompose the complex PDE solution into several simple parts, which decreases the complexity needed to learn each part and boosts generalization. On the other hand, decomposition leads to less training data being available in each subdomain, and hence such model is typically prone to overfitting and may become less generalizable. Empirically, we choose five PDEs to show when XPINNs perform better than, similar to, or worse than PINNs, hence demonstrating and justifying our new theory.

Published in SIAM Journal on Scientific Computing (SISC)

References in corpus (8)

• Flow over an espresso cup: Inferring 3D velocity and pressure fields from tomographic background oriented schlieren videos via physics-informed neural networks
• Exploring Generalization in Deep Learning
• Spectrally-normalized margin bounds for neural networks
• Norm-Based Capacity Control in Neural Networks
• A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations
• On the Banach spaces associated with multi-layer ReLU networks: Function representation, approximation theory and gradient descent dynamics
• Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth
• On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layers

Cited by in corpus (14)

• Tackling the Curse of Dimensionality with Physics-Informed Neural Networks
• Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology
• A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions
• Mixed formulation of physics-informed neural networks for thermo-mechanically coupled systems and heterogeneous domains
• Physics-informed data based neural networks for two-dimensional turbulence
• An unsupervised latent/output physics-informed convolutional-LSTM network for solving partial differential equations using peridynamic differential operator
• Hutchinson Trace Estimation for High-Dimensional and High-Order Physics-Informed Neural Networks
• A Framework Based on Symbolic Regression Coupled with eXtended Physics-Informed Neural Networks for Gray-Box Learning of Equations of Motion from Data
• Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies
• Tackling the Curse of Dimensionality in Fractional and Tempered Fractional PDEs with Physics-Informed Neural Networks
• Neural networks: solving the chemistry of the interstellar medium
• Physics Informed Neural Networks for Modeling of 3D Flow-Thermal Problems with Sparse Domain Data
• Neural Fields in Visual Computing and Beyond
• Non-Asymptotic Stability and Consistency Guarantees for Physics-Informed Neural Networks via Coercive Operator Analysis