Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks

Abstract

Physics-informed neural networks (PINNs) leverage neural-networks to find the solutions of partial differential equation (PDE)-constrained optimization problems with initial conditions and boundary conditions as soft constraints. These soft constraints are often considered to be the sources of the complexity in the training phase of PINNs. Here, we demonstrate that the challenge of training (i) persists even when the boundary conditions are strictly enforced, and (ii) is closely related to the Kolmogorov n-width associated with problems demonstrating transport, convection, traveling waves, or moving fronts. Given this realization, we describe the mechanism underlying the training schemes such as those used in eXtended PINNs (XPINN), curriculum regularization, and sequence-to-sequence learning. For an important category of PDEs, i.e., governed by non-linear convection-diffusion equation, we propose reformulating PINNs on a Lagrangian frame of reference, i.e., LPINNs, as a PDE-informed solution. A parallel architecture with two branches is proposed. One branch solves for the state variables on the characteristics, and the second branch solves for the low-dimensional characteristics curves. The proposed architecture conforms to the causality innate to the convection, and leverages the direction of travel of the information in the domain. Finally, we demonstrate that the loss landscapes of LPINNs are less sensitive to the so-called "complexity" of the problems, compared to those in the traditional PINNs in the Eulerian framework.

15 pages, 12 figures

References in corpus (12)

• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• Uncovering near-wall blood flow from sparse data with physics-informed neural networks
• A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder
• A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks
• Physics-informed neural networks for the shallow-water equations on the sphere
• Memory embedded non-intrusive reduced order modeling of non-ergodic flows
• SVD Perspectives for Augmenting DeepONet Flexibility and Interpretability
• Physics-informed neural networks with hard constraints for inverse design
• Improved Training of Physics-Informed Neural Networks with Model Ensembles
• On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks
• Certified machine learning: A posteriori error estimation for physics-informed neural networks
• A fast dynamic smooth adaptive meshing scheme with applications to compressible flow

Cited by in corpus (8)

• A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions
• Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow
• Calibration-Based ALE Model Order Reduction for Hyperbolic Problems with Self-Similar Travelling Discontinuities
• Gradient Flow Based Phase-Field Modeling Using Separable Neural Networks
• Discontinuity-aware KAN-based physics-informed neural networks
• A Low Rank Neural Representation of Entropy Solutions
• Neural semi-Lagrangian method for high-dimensional advection-diffusion problems
• Automated transport separation using the neural shifted proper orthogonal decomposition