A Priori Analysis of Stable Neural Network Solutions to Numerical PDEs

Abstract

Methods for solving PDEs using neural networks have recently become a very important topic. We provide an a priori error analysis for such methods which is based on the ${\mathcal{K}}_1(\mathbb{D})$-norm of the solution. We show that the resulting constrained optimization problem can be efficiently solved using a greedy algorithm, which replaces stochastic gradient descent. Following this, we show that the error arising from discretizing the energy integrals is bounded both in the deterministic case, i.e. when using numerical quadrature, and also in the stochastic case, i.e. when sampling points to approximate the integrals. In the later case, we use a Rademacher complexity analysis, and in the former we use standard numerical quadrature bounds. This extends existing results to methods which use a general dictionary of functions to learn solutions to PDEs and importantly gives a consistent analysis which incorporates the optimization, approximation, and generalization aspects of the problem. In addition, the Rademacher complexity analysis is simplified and generalized, which enables application to a wide range of problems.

arXiv commentThis paper has been merged with arXiv:2107.04466

References in corpus (15)

• DeepXDE: A deep learning library for solving differential equations
• Solving high-dimensional partial differential equations using deep learning
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Approximation and learning by greedy algorithms
• Multipole Graph Neural Operator for Parametric Partial Differential Equations
• Convergence of the Deep BSDE Method for Coupled FBSDEs
• When and why PINNs fail to train: A neural tangent kernel perspective
• A Priori Estimates of the Population Risk for Two-layer Neural Networks
• The Finite Neuron Method and Convergence Analysis
• A Comparative Analysis of the Optimization and Generalization Property of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics
• Convergence Rate Analysis for Deep Ritz Method
• Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory
• Error estimates of residual minimization using neural networks for linear PDEs
• A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations

Cited by in corpus (6)

• A priori and a posteriori error estimates for the Deep Ritz method applied to the Laplace and Stokes problem
• Imaging Conductivity from Current Density Magnitude using Neural Networks
• On the Representation of Solutions to Elliptic PDEs in Barron Spaces
• Solving PDEs on Unknown Manifolds with Machine Learning
• Certified machine learning: Rigorous a posteriori error bounds for PDE defined PINNs
• Deep Ritz-Finite Element methods: Neural Network Methods trained with Finite Elements