CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method

Abstract

In this study, novel physics-informed neural network (PINN) methods for coupling neighboring support points and their derivative terms which are obtained by automatic differentiation (AD), are proposed to allow efficient training with improved accuracy. The computation of differential operators required for PINNs loss evaluation at collocation points are conventionally obtained via AD. Although AD has the advantage of being able to compute the exact gradients at any point, such PINNs can only achieve high accuracies with large numbers of collocation points, otherwise they are prone to optimizing towards unphysical solution. To make PINN training fast, the dual ideas of using numerical differentiation (ND)-inspired method and coupling it with AD are employed to define the loss function. The ND-based formulation for training loss can strongly link neighboring collocation points to enable efficient training in sparse sample regimes, but its accuracy is restricted by the interpolation scheme. The proposed coupled-automatic-numerical differentiation framework, labeled as can-PINN, unifies the advantages of AD and ND, providing more robust and efficient training than AD-based PINNs, while further improving accuracy by up to 1-2 orders of magnitude relative to ND-based PINNs. For a proof-of-concept demonstration of this can-scheme to fluid dynamic problems, two numerical-inspired instantiations of can-PINN schemes for the convection and pressure gradient terms were derived to solve the incompressible Navier-Stokes (N-S) equations. The superior performance of can-PINNs is demonstrated on several challenging problems, including the flow mixing phenomena, lid driven flow in a cavity, and channel flow over a backward facing step. The results reveal that for challenging problems like these, can-PINNs can consistently achieve very good accuracy whereas conventional AD-based PINNs fail.

arXiv comment25 pages, 20 figures

References in corpus (32)

• TensorFlow: A system for large-scale machine learning
• DeepXDE: A deep learning library for solving differential equations
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• NSFnets (Navier-Stokes Flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations
• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Physics-informed neural networks for inverse problems in nano-optics and metamaterials
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks
• PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain
• Physics-Informed Multi-LSTM Networks for Metamodeling of Nonlinear Structures
• Numerical Solutions of 2-D Steady Incompressible Driven Cavity Flow at High Reynolds Numbers
• A Physics-Informed Machine Learning Approach for Solving Heat Transfer Equation in Advanced Manufacturing and Engineering Applications
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• SciANN: A Keras/Tensorflow wrapper for scientific computations and physics-informed deep learning using artificial neural networks
• Deep Learning of Vortex Induced Vibrations
• Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks
• Efficient training of physics-informed neural networks via importance sampling
• Physics-Informed Neural Network for Modelling the Thermochemical Curing Process of Composite-Tool Systems During Manufacture
• Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems
• On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs
• PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs
• Understanding and mitigating gradient pathologies in physics-informed neural networks
• Artificial Neural Network Methods in Quantum Mechanics
• A nonlocal physics-informed deep learning framework using the peridynamic differential operator
• Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism
• Learning in Sinusoidal Spaces with Physics-Informed Neural Networks
• Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis
• Teaching the Incompressible Navier-Stokes Equations to Fast Neural Surrogate Models in 3D
• Can Transfer Neuroevolution Tractably Solve Your Differential Equations?

Cited by in corpus (22)

• Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios
• Deep Learning in Deterministic Computational Mechanics
• Mixed formulation of physics-informed neural networks for thermo-mechanically coupled systems and heterogeneous domains
• Learning in Sinusoidal Spaces with Physics-Informed Neural Networks
• A solver for subsonic flow around airfoils based on physics-informed neural networks and mesh transformation
• RBF-MGN:Solving spatiotemporal PDEs with Physics-informed Graph Neural Network
• Predicting nonlinear dynamics of optical solitons in optical fiber via the SCPINN
• Fully probabilistic deep models for forward and inverse problems in parametric PDEs
• FFV-PINN: A Fast Physics-Informed Neural Network with Simplified Finite Volume Discretization and Residual Correction
• Deep Ritz Method with Adaptive Quadrature for Linear Elasticity
• A Spectral-based Physics-informed Finite Operator Learning for Prediction of Mechanical Behavior of Microstructures
• Physics-informed Neural Networks for Heterogeneous Poroelastic Media
• On the locality of local neural operator in learning fluid dynamics
• Physics-Informed Boundary Integral Networks (PIBI-Nets): A Data-Driven Approach for Solving Partial Differential Equations
• Finite-difference-informed graph network for solving steady-state incompressible flows on block-structured grids
• LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry
• Multi-Fidelity Machine Learning Applied to Steady Fluid Flows
• Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities
• Physics-informed Neural Networks approach to solve the Blasius function
• Numerical Differentiation-based Electrophysiology-Aware Adaptive ResNet for Inverse ECG Modeling
• Fast meta-solvers for 3D complex-shape scatterers using neural operators trained on a non-scattering problem
• Multi-patch isogeometric neural solver for partial differential equations on computer-aided design domains