LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data

Abstract

Neural operators, as a powerful approximation to the non-linear operators between infinite-dimensional function spaces, have proved to be promising in accelerating the solution of partial differential equations (PDE). However, it requires a large amount of simulated data, which can be costly to collect. This can be avoided by learning physics from the physics-constrained loss, which we refer to it as mean squared residual (MSR) loss constructed by the discretized PDE. We investigate the physical information in the MSR loss, which we called long-range entanglements, and identify the challenge that the neural network requires the capacity to model the long-range entanglements in the spatial domain of the PDE, whose patterns vary in different PDEs. To tackle the challenge, we propose LordNet, a tunable and efficient neural network for modeling various entanglements. Inspired by the traditional solvers, LordNet models the long-range entanglements with a series of matrix multiplications, which can be seen as the low-rank approximation to the general fully-connected layers and extracts the dominant pattern with reduced computational cost. The experiments on solving Poisson's equation and (2D and 3D) Navier-Stokes equation demonstrate that the long-range entanglements from the MSR loss can be well modeled by the LordNet, yielding better accuracy and generalization ability than other neural networks. The results show that the Lordnet can be $40\times$ faster than traditional PDE solvers. In addition, LordNet outperforms other modern neural network architectures in accuracy and efficiency with the smallest parameter size.

References in corpus (18)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• MLP-Mixer: An all-MLP Architecture for Vision
• NSFnets (Navier-Stokes Flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations
• Fourier Neural Operator for Parametric Partial Differential Equations
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networks
• Learning data driven discretizations for partial differential equations
• Deep neural network solution of the electronic Schrödinger equation
• Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks
• Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
• Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks
• Transformers for Modeling Physical Systems
• Message Passing Neural PDE Solvers
• Teaching the Incompressible Navier-Stokes Equations to Fast Neural Surrogate Models in 3D
• Learning Neural PDE Solvers with Convergence Guarantees
• Clifford Neural Layers for PDE Modeling

Cited by in corpus (1)

• Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations