A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions

Abstract

Physics-informed neural networks (PINNs) as a means of solving partial differential equations (PDE) have garnered much attention in the Computational Science and Engineering (CS&E) world. However, a recent topic of interest is exploring various training (i.e., optimization) challenges - in particular, arriving at poor local minima in the optimization landscape results in a PINN approximation giving an inferior, and sometimes trivial, solution when solving forward time-dependent PDEs with no data. This problem is also found in, and in some sense more difficult, with domain decomposition strategies such as temporal decomposition using XPINNs. We furnish examples and explanations for different training challenges, their cause, and how they relate to information propagation and temporal decomposition. We then propose a new stacked-decomposition method that bridges the gap between time-marching PINNs and XPINNs. We also introduce significant computational speed-ups by using transfer learning concepts to initialize subnetworks in the domain and loss tolerance-based propagation for the subdomains. Finally, we formulate a new time-sweeping collocation point algorithm inspired by the previous PINNs causality literature, which our framework can still describe, and provides a significant computational speed-up via reduced-cost collocation point segmentation. The proposed methods form our unified framework, which overcomes training challenges in PINNs and XPINNs for time-dependent PDEs by respecting the causality in multiple forms and improving scalability by limiting the computation required per optimization iteration. Finally, we provide numerical results for these methods on baseline PDE problems for which unmodified PINNs and XPINNs struggle to train.

References in corpus (14)

• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Physics-informed neural networks for inverse problems in nano-optics and metamaterials
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks
• Physics-informed neural networks for the shallow-water equations on the sphere
• When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?
• A physics-informed neural network for quantifying the microstructure properties of polycrystalline Nickel using ultrasound data
• Deep learning of free boundary and Stefan problems
• Uncovering turbulent plasma dynamics via deep learning from partial observations
• Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks

Cited by in corpus (6)

• Physics-Informed Neural Networks for Parametric Compressible Euler Equations
• Physics-informed neural networks modeling for systems with moving immersed boundaries: application to an unsteady flow past a plunging foil
• An extrapolation-driven network architecture for physics-informed deep learning
• Sequential learning based PINNs to overcome temporal domain complexities in unsteady flow past flapping wings
• Regime-Aware Time Weighting for Physics-Informed Neural Networks
• Optimal time sampling in physics-informed neural networks