A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks

Abstract

Physics-informed neural networks (PINNs) have shown to be an effective tool for solving forward and inverse problems of partial differential equations (PDEs). PINNs embed the PDEs into the loss of the neural network, and this PDE loss is evaluated at a set of scattered residual points. The distribution of these points are highly important to the performance of PINNs. However, in the existing studies on PINNs, only a few simple residual point sampling methods have mainly been used. Here, we present a comprehensive study of two categories of sampling: non-adaptive uniform sampling and adaptive nonuniform sampling. We consider six uniform sampling, including (1) equispaced uniform grid, (2) uniformly random sampling, (3) Latin hypercube sampling, (4) Halton sequence, (5) Hammersley sequence, and (6) Sobol sequence. We also consider a resampling strategy for uniform sampling. To improve the sampling efficiency and the accuracy of PINNs, we propose two new residual-based adaptive sampling methods: residual-based adaptive distribution (RAD) and residual-based adaptive refinement with distribution (RAR-D), which dynamically improve the distribution of residual points based on the PDE residuals during training. Hence, we have considered a total of 10 different sampling methods, including six non-adaptive uniform sampling, uniform sampling with resampling, two proposed adaptive sampling, and an existing adaptive sampling. We extensively tested the performance of these sampling methods for four forward problems and two inverse problems in many setups. Our numerical results presented in this study are summarized from more than 6000 simulations of PINNs. We show that the proposed adaptive sampling methods of RAD and RAR-D significantly improve the accuracy of PINNs with fewer residual points. The results obtained in this study can also be used as a practical guideline in choosing sampling methods.

References in corpus (12)

• Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
• Self-adaptive loss balanced Physics-informed neural networks for the incompressible Navier-Stokes equations
• Efficient training of physics-informed neural networks via importance sampling
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks
• Meta-learning PINN loss functions
• Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism
• Residual-based adaptivity for two-phase flow simulation in porous media using Physics-informed Neural Networks
• Improved Training of Physics-Informed Neural Networks with Model Ensembles
• State-of-the-Art Review of Design of Experiments for Physics-Informed Deep Learning
• RANG: A Residual-based Adaptive Node Generation Method for Physics-Informed Neural Networks
• Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Method

Cited by in corpus (56)

• Residual-based attention in physics-informed neural networks
• Multi-Objective Loss Balancing for Physics-Informed Deep Learning
• Reliable extrapolation of deep neural operators informed by physics or sparse observations
• Physics-informed radial basis network (PIRBN): A local approximating neural network for solving nonlinear PDEs
• Active Learning and Bayesian Optimization: a Unified Perspective to Learn with a Goal
• Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?
• Physics-informed neural network methods based on Miura transformations and discovery of new localized wave solutions
• A solver for subsonic flow around airfoils based on physics-informed neural networks and mesh transformation
• Adaptive Training of Grid-Dependent Physics-Informed Kolmogorov-Arnold Networks
• Efficient physics-informed neural networks using hash encoding
• Data vs. Physics: The Apparent Pareto Front of Physics-Informed Neural Networks
• Learning Specialized Activation Functions for Physics-informed Neural Networks
• GAS: A Gaussian Mixture Distribution-Based Adaptive Sampling Method for PINNs
• Improved Training of Physics-Informed Neural Networks with Model Ensembles
• Applications of Scientific Machine Learning for the Analysis of Functionally Graded Porous Beams
• Physics-Informed Neural Network Method for Parabolic Differential Equations with Sharply Perturbed Initial Conditions
• PPDONet: Deep Operator Networks for Fast Prediction of Steady-State Solutions in Disk-Planet Systems
• Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks
• Physics-informed neural networks for unsteady incompressible flows with time-dependent moving boundaries
• Physics-informed neural networks for gravity currents reconstruction from limited data
• Physics-Informed Deep Learning of Rate-and-State Fault Friction
• Physics-informed neural networks modeling for systems with moving immersed boundaries: application to an unsteady flow past a plunging foil
• Fourier neural operator for learning solutions to macroscopic traffic flow models: Application to the forward and inverse problems
• Adapting Physics-Informed Neural Networks to Improve ODE Optimization in Mosquito Population Dynamics
• Physics-Informed Holomorphic Neural Networks (PIHNNs): Solving Linear Elasticity Problems
• An Advanced Physics-Informed Neural Operator for Comprehensive Design Optimization of Highly-Nonlinear Systems: An Aerospace Composites Processing Case Study
• A multifidelity approach to continual learning for physical systems
• The Hard-Constraint PINNs for Interface Optimal Control Problems
• Physics Informed Neural Networks for Modeling of 3D Flow-Thermal Problems with Sparse Domain Data
• Gradient and Uncertainty Enhanced Sequential Sampling for Global Fit
• Learning in PINNs: Phase transition, diffusion equilibrium, and generalization
• Approximately well-balanced Discontinuous Galerkin methods using bases enriched with Physics-Informed Neural Networks
• Geometry-aware framework for deep energy method: an application to structural mechanics with hyperelastic materials
• A Physics-Informed Deep Learning Description of Knudsen Layer Reactivity Reduction
• LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry
• Physics-informed Deep Learning to Solve Three-dimensional Terzaghi Consolidation Equation: Forward and Inverse Problems
• Scientific machine learning in Hydrology: a unified perspective
• Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network
• Investigating Guiding Information for Adaptive Collocation Point Sampling in PINNs
• Learning Feynman integrals from differential equations with neural networks
• Training Deep Physics-Informed Kolmogorov-Arnold Networks
• A PINN Methodology for Temperature Field Reconstruction in the PIV Measurement Plane: Case of Rayleigh-Bénard Convection
• Learning quantities of interest from parametric PDEs: An efficient neural-weighted Minimal Residual approach
• Neural Hawkes: Non-Parametric Estimation in High Dimension and Causality Analysis in Cryptocurrency Markets
• Gradient Flow Based Phase-Field Modeling Using Separable Neural Networks
• Spectral Informed Neural Network: An Efficient and Low-Memory PINN
• Fixed-budget online adaptive learning for physics-informed neural networks. Towards parameterized problem inference
• Neural semi-Lagrangian method for high-dimensional advection-diffusion problems
• Active learning with physics-informed neural networks for optimal sensor placement in deep tunneling through transversely isotropic elastic rocks
• Critical Sampling for Robust Evolution Operator Learning of Unknown Dynamical Systems
• Towards asteroseismology of neutron stars with physics-informed neural networks
• Spatio-Temporal Uncertainty-Modulated Physics-Informed Neural Networks for Solving Hyperbolic Conservation Laws with Strong Shocks
• Transforming physics-informed machine learning to convex optimization
• Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networks
• Quasi-Random Physics-informed Neural Networks
• A Variational Kolosov--Muskhelishvili Network for Elasticity and Fracture