Adversarial Multi-task Learning Enhanced Physics-informed Neural Networks for Solving Partial Differential Equations

Abstract

Recently, researchers have utilized neural networks to accurately solve partial differential equations (PDEs), enabling the mesh-free method for scientific computation. Unfortunately, the network performance drops when encountering a high nonlinearity domain. To improve the generalizability, we introduce the novel approach of employing multi-task learning techniques, the uncertainty-weighting loss and the gradients surgery, in the context of learning PDE solutions. The multi-task scheme exploits the benefits of learning shared representations, controlled by cross-stitch modules, between multiple related PDEs, which are obtainable by varying the PDE parameterization coefficients, to generalize better on the original PDE. Encouraging the network pay closer attention to the high nonlinearity domain regions that are more challenging to learn, we also propose adversarial training for generating supplementary high-loss samples, similarly distributed to the original training distribution. In the experiments, our proposed methods are found to be effective and reduce the error on the unseen data points as compared to the previous approaches in various PDE examples, including high-dimensional stochastic PDEs.

arXiv commentAccepted by the International Joint Conference on Neural Networks (IJCNN) 2021, Oral presentation

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Explaining and Harnessing Adversarial Examples
• Practical Bayesian Optimization of Machine Learning Algorithms
• Intriguing properties of neural networks
• Machine learning and the physical sciences
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Solving Fokker-Planck equation using deep learning
• A Discussion on Solving Partial Differential Equations using Neural Networks
• Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations
• Solving Nonlinear and High-Dimensional Partial Differential Equations via Deep Learning
• Towards Robust and Stable Deep Learning Algorithms for Forward Backward Stochastic Differential Equations

Cited by in corpus (6)

• Learning in Sinusoidal Spaces with Physics-Informed Neural Networks
• Noise-aware Physics-informed Machine Learning for Robust PDE Discovery
• Spectrally Adapted Physics-Informed Neural Networks for Solving Unbounded Domain Problems
• A Tutorial on the Use of Physics-Informed Neural Networks to Compute the Spectrum of Quantum Systems
• LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry
• Adaptive Uncertainty-Guided Model Selection for Data-Driven PDE Discovery