On quadrature rules for solving Partial Differential Equations using Neural Networks

Abstract

Neural Networks have been widely used to solve Partial Differential Equations. These methods require to approximate definite integrals using quadrature rules. Here, we illustrate via 1D numerical examples the quadrature problems that may arise in these applications and propose different alternatives to overcome them, namely: Monte Carlo methods, adaptive integration, polynomial approximations of the Neural Network output, and the inclusion of regularization terms in the loss. We also discuss the advantages and limitations of each proposed alternative. We advocate the use of Monte Carlo methods for high dimensions (above 3 or 4), and adaptive integration or polynomial approximations for low dimensions (3 or below). The use of regularization terms is a mathematically elegant alternative that is valid for any spacial dimension, however, it requires certain regularity assumptions on the solution and complex mathematical analysis when dealing with sophisticated Neural Networks.

References in corpus (19)

• DeepXDE: A deep learning library for solving differential equations
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications
• Solving high-dimensional partial differential equations using deep learning
• fPINNs: Fractional Physics-Informed Neural Networks
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Deep Audio-Visual Speech Recognition
• Isogeometric analysis: an overview and computer implementation aspects
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• A Study on Overfitting in Deep Reinforcement Learning
• Data-Driven Deep Learning of Partial Differential Equations in Modal Space
• On the Spectral Bias of Neural Networks
• When and why PINNs fail to train: A neural tangent kernel perspective
• Deep least-squares methods: an unsupervised learning-based numerical method for solving elliptic PDEs
• Introduction to Monte Carlo methods
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs
• Data-Driven Finite Elements Methods: Machine Learning Acceleration of Goal-Oriented Computations

Cited by in corpus (7)

• Deep Learning in Deterministic Computational Mechanics
• Finite element interpolated neural networks for solving forward and inverse problems
• Deep Ritz Method with Adaptive Quadrature for Linear Elasticity
• A Deep Double Ritz Method (D${}^2$RM) for solving Partial Differential Equations using Neural Networks
• Compatible finite element interpolated neural networks
• Learning quantities of interest from parametric PDEs: An efficient neural-weighted Minimal Residual approach
• An $r$-adaptive finite element method using neural networks for parametric self-adjoint elliptic problem