Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies

Abstract

We propose to enhance the training of physics-informed neural networks (PINNs). To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain-decomposition framework, where the parameters of the network are decomposed in a layer-wise manner. Through a series of numerical experiments, we demonstrate that both, additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer, while providing more accurate solutions of the underlying partial differential equations. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism.

23 pages, 7 figures

References in corpus (4)

• Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
• Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology
• When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?
• Nonlinear Field-split Preconditioners for Solving Monolithic Phase-field Models of Brittle Fracture

Cited by in corpus (8)

• Residual-based attention in physics-informed neural networks
• Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?
• Learning in PINNs: Phase transition, diffusion equilibrium, and generalization
• Preconditioned FEM-based Neural Networks for Solving Incompressible Fluid Flows and Related Inverse Problems
• An operator preconditioning perspective on training in physics-informed machine learning
• A Nonoverlapping Domain Decomposition Method for Extreme Learning Machines: Elliptic Problems
• Two-level overlapping additive Schwarz preconditioner for training scientific machine learning applications
• Deep Domain Decomposition Method for Solving the Variational Inequality Problems