On the convergence of PINNs
arXiv:2305.01240
Abstract
Physics-informed neural networks (PINNs) are a promising approach that combines the power of neural networks with the interpretability of physical modeling. PINNs have shown good practical performance in solving partial differential equations (PDEs) and in hybrid modeling scenarios, where physical models enhance data-driven approaches. However, it is essential to establish their theoretical properties in order to fully understand their capabilities and limitations. In this study, we highlight that classical training of PINNs can suffer from systematic overfitting. This problem can be addressed by adding a ridge regularization to the empirical risk, which ensures that the resulting estimator is risk-consistent for both linear and nonlinear PDE systems. However, the strong convergence of PINNs to a solution satisfying the physical constraints requires a more involved analysis using tools from functional analysis and calculus of variations. In particular, for linear PDE systems, an implementable Sobolev-type regularization allows to reconstruct a solution that not only achieves statistical accuracy but also maintains consistency with the underlying physics.
References in corpus (19)
- Decoupled Weight Decay Regularization
- On Calibration of Modern Neural Networks
- Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
- Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
- Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport
- Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge
- Solving the acoustic VTI wave equation using physics-informed neural networks
- Physics Informed Neural Networks for Control Oriented Thermal Modeling of Buildings
- A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic
- Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications
- Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis
- Physics-Informed and Hybrid Machine Learning in Additive Manufacturing: Application to Fused Filament Fabrication
- Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networks
- Physics-Informed Neural Network Super Resolution for Advection-Diffusion Models
- Towards Optimally Weighted Physics-Informed Neural Networks in Ocean Modelling
- Sobolev Norm Learning Rates for Regularized Least-Squares Algorithm
- A Data-driven Multi-fidelity Physics-informed Learning Framework for Smart Manufacturing: A Composites Processing Case Study
- On Sobolev extension domains in
- Error Analysis of Physics-Informed Neural Networks for Approximating Dynamic PDEs of Second Order in Time