SPINN: Sparse, Physics-based, and partially Interpretable Neural Networks for PDEs

Abstract

We introduce a class of Sparse, Physics-based, and partially Interpretable Neural Networks (SPINN) for solving ordinary and partial differential equations (PDEs). By reinterpreting a traditional meshless representation of solutions of PDEs we develop a class of sparse neural network architectures that are partially interpretable. The SPINN model we propose here serves as a seamless bridge between two extreme modeling tools for PDEs, namely dense neural network based methods like Physics Informed Neural Networks (PINNs) and traditional mesh-free numerical methods, thereby providing a novel means to develop a new class of hybrid algorithms that build on the best of both these viewpoints. A unique feature of the SPINN model that distinguishes it from other neural network based approximations proposed earlier is that it is (i) interpretable, in a particular sense made precise in the work, and (ii) sparse in the sense that it has much fewer connections than typical dense neural networks used for PDEs. Further, the SPINN algorithm implicitly encodes mesh adaptivity and is able to handle discontinuities in the solutions. In addition, we demonstrate that Fourier series representations can also be expressed as a special class of SPINN and propose generalized neural network analogues of Fourier representations. We illustrate the utility of the proposed method with a variety of examples involving ordinary differential equations, elliptic, parabolic, hyperbolic and nonlinear partial differential equations, and an example in fluid dynamics.

arXiv comment58 pages, 32 figures, 1 table

References in corpus (26)

• Adam: A Method for Stochastic Optimization
• Array Programming with NumPy
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• DeepXDE: A deep learning library for solving differential equations
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications
• Solving high-dimensional partial differential equations using deep learning
• Fast Graph Representation Learning with PyTorch Geometric
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• Mayavi: a package for 3D visualization of scientific data
• PDE-Net: Learning PDEs from Data
• Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains
• ReLU Deep Neural Networks and Linear Finite Elements
• Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems
• Deep least-squares methods: an unsupervised learning-based numerical method for solving elliptic PDEs
• A mesh-free method for interface problems using the deep learning approach
• PyClaw: Accessible, Extensible, Scalable Tools for Wave Propagation Problems
• Entropically Damped Artificial Compressibility for SPH
• Physics-informed neural networks with hard constraints for inverse design
• Machine Learning from a Continuous Viewpoint
• automan: a simple, Python-based, automation framework for numerical computing
• Deep Euler method: solving ODEs by approximating the local truncation error of the Euler method
• A Mesh-free Method Using Piecewise Deep Neural Network for Elliptic Interface Problems
• Implicit bias with Ritz-Galerkin method in understanding deep learning for solving PDEs

Cited by in corpus (8)

• Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion
• Physics-informed radial basis network (PIRBN): A local approximating neural network for solving nonlinear PDEs
• Learning Stochastic Dynamics with Statistics-Informed Neural Network
• Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks
• LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry
• Learning Domain-Independent Green's Function For Elliptic Partial Differential Equations
• Numerical simulation of transient heat conduction with moving heat source using Physics Informed Neural Networks
• Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data