Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networks

Abstract

Physics-informed Neural Networks (PINNs) have been shown to be effective in solving partial differential equations by capturing the physics induced constraints as a part of the training loss function. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE. It also shows how physical regularizations based on continuity criteria and conservation laws fail to address this issue and rather introduce problems of their own causing the deep network to converge to a physics-obeying local minimum instead of the global minimum. We introduce Gaussian Process (GP) based smoothing that recovers the performance of a PINN and promises a robust architecture against noise/errors in measurements. Additionally, we illustrate an inexpensive method of quantifying the evolution of uncertainty based on the variance estimation of GPs on boundary data. Robust PINN performance is also shown to be achievable by choice of sparse sets of inducing points based on sparsely induced GPs. We demonstrate the performance of our proposed methods and compare the results from existing benchmark models in literature for time-dependent Schrödinger and Burgers' equations.

arXiv commentAccepted at Machine Learning: Science and Technology

References in corpus (17)

• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• fPINNs: Fractional Physics-Informed Neural Networks
• Machine learning for molecular simulation
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• Neural Ordinary Differential Equations
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Structure Discovery in Nonparametric Regression through Compositional Kernel Search
• On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs
• Variational Physics-Informed Neural Networks For Solving Partial Differential Equations
• Additive Gaussian Processes
• Constructing exact representations of quantum many-body systems with deep neural networks
• ODE${}^2$VAE: Deep generative second order ODEs with Bayesian neural networks
• Extrapolating quantum observables with machine learning: Inferring multiple phase transitions from properties of a single phase
• Adversarial Robustness Guarantees for Classification with Gaussian Processes
• Sample-efficient reinforcement learning using deep Gaussian processes

Cited by in corpus (4)

• Spectrally Adapted Physics-Informed Neural Networks for Solving Unbounded Domain Problems
• A multifidelity approach to continual learning for physical systems
• Adaptive Uncertainty-Guided Model Selection for Data-Driven PDE Discovery
• On the convergence of PINNs