Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?

Abstract

This study investigates the potential accuracy boundaries of physics-informed neural networks, contrasting their approach with previous similar works and traditional numerical methods. We find that selecting improved optimization algorithms significantly enhances the accuracy of the results. Simple modifications to the loss function may also improve precision, offering an additional avenue for enhancement. Despite optimization algorithms having a greater impact on convergence than adjustments to the loss function, practical considerations often favor tweaking the latter due to ease of implementation. On a global scale, the integration of an enhanced optimizer and a marginally adjusted loss function enables a reduction in the loss function by several orders of magnitude across diverse physical problems. Consequently, our results obtained using compact networks (typically comprising 2 or 3 layers of 20-30 neurons) achieve accuracies comparable to finite difference schemes employing thousands of grid points. This study encourages the continued advancement of PINNs and associated optimization techniques for broader applications across various fields.

63 pages, 25 figures. This is the author-accepted manuscript of the paper published in Journal of Computational Physics

References in corpus (10)

• A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
• Residual-based attention in physics-informed neural networks
• Tackling the Curse of Dimensionality with Physics-Informed Neural Networks
• Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology
• A physics-informed neural network for quantifying the microstructure properties of polycrystalline Nickel using ultrasound data
• Physics-informed radial basis network (PIRBN): A local approximating neural network for solving nonlinear PDEs
• A practical PINN framework for multi-scale problems with multi-magnitude loss terms
• Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies
• Solving the Teukolsky equation with physics-informed neural networks
• Modelling Force-Free Neutron Star Magnetospheres using Physics-Informed Neural Networks

Cited by in corpus (7)

• Magnetic, thermal and rotational evolution of isolated neutron stars
• Physics-informed Deep Learning to Solve Three-dimensional Terzaghi Consolidation Equation: Forward and Inverse Problems
• General-relativistic magnetar magnetospheres in 3D with physics-informed neural networks
• An approximate Riemann solver approach in Physics-Informed Neural Networks for hyperbolic conservation laws
• Discontinuity-aware KAN-based physics-informed neural networks
• Unfitted finite element interpolated neural networks
• Neural network methods for Neumann series problems of Perron-Frobenius operators