Hybrid FEM-NN models: Combining artificial neural networks with the finite element method

Abstract

We present a methodology combining neural networks with physical principle constraints in the form of partial differential equations (PDEs). The approach allows to train neural networks while respecting the PDEs as a strong constraint in the optimisation as apposed to making them part of the loss function. The resulting models are discretised in space by the finite element method (FEM). The method applies to both stationary and transient as well as linear/nonlinear PDEs. We describe implementation of the approach as an extension of the existing FEM framework FEniCS and its algorithmic differentiation tool dolfin-adjoint. Through series of examples we demonstrate capabilities of the approach to recover coefficients and missing PDE operators from observations. Further, the proposed method is compared with alternative methodologies, namely, physics informed neural networks and standard PDE-constrained optimisation. Finally, we demonstrate the method on a complex cardiac cell model problem using deep neural networks.

References in corpus (25)

• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Turbulence Modeling in the Age of Data
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• Fourier Neural Operator for Parametric Partial Differential Equations
• DOLFIN: Automated Finite Element Computing
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• Deep Learning: A Critical Appraisal
• PDE-Net: Learning PDEs from Data
• Variational Physics-Informed Neural Networks For Solving Partial Differential Equations
• Understanding and mitigating gradient pathologies in physics-informed neural networks
• Learning Constitutive Relations from Indirect Observations Using Deep Neural Networks
• When and why PINNs fail to train: A neural tangent kernel perspective
• Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks
• Learning Dynamical Systems from Partial Observations
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs
• Neural network augmented inverse problems for PDEs
• ADCME: Learning Spatially-varying Physical Fields using Deep Neural Networks
• Inverse Modeling of Viscoelasticity Materials using Physics Constrained Learning
• Exposing and exploiting structure: optimal code generation for high-order finite element methods
• Solving Inverse Problems in Steady-State Navier-Stokes Equations using Deep Neural Networks
• Revisiting Calderon's Problem

Cited by in corpus (17)

• Differentiable modeling to unify machine learning and physical models and advance Geosciences
• Integrated Finite Element Neural Network (I-FENN) for non-local continuum damage mechanics
• Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics
• $\Delta$-PINNs: physics-informed neural networks on complex geometries
• Finite element interpolated neural networks for solving forward and inverse problems
• A finite element-based physics-informed operator learning framework for spatiotemporal partial differential equations on arbitrary domains
• DNN-MG: A Hybrid Neural Network/Finite Element Method with Applications to 3D Simulations of the Navier-Stokes Equations
• I-FENN with Temporal Convolutional Networks: expediting the load-history analysis of non-local gradient damage propagation
• Fixed-energy inverse scattering with radial basis function neural networks and its application to neutron-alpha interactions
• Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling
• LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry
• Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation
• Preconditioned FEM-based Neural Networks for Solving Incompressible Fluid Flows and Related Inverse Problems
• Compatible finite element interpolated neural networks
• Sobolev neural network with residual weighting as a surrogate in linear and non-linear mechanics
• Online learning in idealized ocean gyres
• Non-intrusive model combination for learning dynamical systems