Hierarchical Learning to Solve Partial Differential Equations Using Physics-Informed Neural Networks

Abstract

The neural network-based approach to solving partial differential equations has attracted considerable attention due to its simplicity and flexibility in representing the solution of the partial differential equation. In training a neural network, the network learns global features corresponding to low-frequency components while high-frequency components are approximated at a much slower rate. For a class of equations in which the solution contains a wide range of scales, the network training process can suffer from slow convergence and low accuracy due to its inability to capture the high-frequency components. In this work, we propose a hierarchical approach to improve the convergence rate and accuracy of the neural network solution to partial differential equations. The proposed method comprises multi-training levels in which a newly introduced neural network is guided to learn the residual of the previous level approximation. By the nature of neural networks' training process, the high-level correction is inclined to capture the high-frequency components. We validate the efficiency and robustness of the proposed hierarchical approach through a suite of linear and nonlinear partial differential equations.

arXiv comment20 pages, 12 figures

References in corpus (16)

• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
• Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains
• Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks
• Data-driven discovery of PDEs in complex datasets
• On the Spectral Bias of Neural Networks
• When and why PINNs fail to train: A neural tangent kernel perspective
• Physics-informed neural networks (PINNs) for fluid mechanics: A review
• A Derivative-Free Method for Solving Elliptic Partial Differential Equations with Deep Neural Networks
• Multi-scale Deep Neural Network (MscaleDNN) Methods for Oscillatory Stokes Flows in Complex Domains
• Multi-scale Deep Neural Networks for Solving High Dimensional PDEs
• Multiscale numerical methods for passive advection-diffusion in incompressible turbulent flow fields