Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?

Abstract

This study investigates the potential accuracy boundaries of physics-informed neural networks, contrasting their approach with previous similar works and traditional numerical methods. We find that selecting improved optimization algorithms significantly enhances the accuracy of the results. Simple modifications to the loss function may also improve precision, offering an additional avenue for enhancement. Despite optimization algorithms having a greater impact on convergence than adjustments to the loss function, practical considerations often favor tweaking the latter due to ease of implementation. On a global scale, the integration of an enhanced optimizer and a marginally adjusted loss function enables a reduction in the loss function by several orders of magnitude across diverse physical problems. Consequently, our results obtained using compact networks (typically comprising 2 or 3 layers of 20-30 neurons) achieve accuracies comparable to finite difference schemes employing thousands of grid points. This study encourages the continued advancement of PINNs and associated optimization techniques for broader applications across various fields.

63 pages, 25 figures. This is the author-accepted manuscript of the paper published in Journal of Computational Physics

References in corpus (23)

• NSFnets (Navier-Stokes Flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
• Physics-informed neural networks for inverse problems in nano-optics and metamaterials
• Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks
• Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent
• Residual-based attention in physics-informed neural networks
• A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks
• Tackling the Curse of Dimensionality with Physics-Informed Neural Networks
• Physics Informed Neural Networks for Simulating Radiative Transfer
• Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology
• A physics-informed neural network for quantifying the microstructure properties of polycrystalline Nickel using ultrasound data
• A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecture
• Physics-informed radial basis network (PIRBN): A local approximating neural network for solving nonlinear PDEs
• Uncovering turbulent plasma dynamics via deep learning from partial observations
• A practical PINN framework for multi-scale problems with multi-magnitude loss terms
• Solving the Teukolsky equation with physics-informed neural networks
• A nonlinear Schrödinger equation for gravity-capillary water waves on arbitrary depth with constant vorticity: Part I
• Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies
• The force-free twisted magnetosphere of a neutron star
• The KdV hierarchy in optics
• Modelling Force-Free Neutron Star Magnetospheres using Physics-Informed Neural Networks

Cited by in corpus (7)

• Magnetic, thermal and rotational evolution of isolated neutron stars
• Physics-informed Deep Learning to Solve Three-dimensional Terzaghi Consolidation Equation: Forward and Inverse Problems
• General-relativistic magnetar magnetospheres in 3D with physics-informed neural networks
• An approximate Riemann solver approach in Physics-Informed Neural Networks for hyperbolic conservation laws
• Discontinuity-aware KAN-based physics-informed neural networks
• Unfitted finite element interpolated neural networks
• Neural network methods for Neumann series problems of Perron-Frobenius operators