A Priori Analysis of Stable Neural Network Solutions to Numerical PDEs

Abstract

Methods for solving PDEs using neural networks have recently become a very important topic. We provide an a priori error analysis for such methods which is based on the ${\mathcal{K}}_1(\mathbb{D})$-norm of the solution. We show that the resulting constrained optimization problem can be efficiently solved using a greedy algorithm, which replaces stochastic gradient descent. Following this, we show that the error arising from discretizing the energy integrals is bounded both in the deterministic case, i.e. when using numerical quadrature, and also in the stochastic case, i.e. when sampling points to approximate the integrals. In the later case, we use a Rademacher complexity analysis, and in the former we use standard numerical quadrature bounds. This extends existing results to methods which use a general dictionary of functions to learn solutions to PDEs and importantly gives a consistent analysis which incorporates the optimization, approximation, and generalization aspects of the problem. In addition, the Rademacher complexity analysis is simplified and generalized, which enables application to a wide range of problems.

This paper has been merged with arXiv:2107.04466

References in corpus (8)

• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Approximation and learning by greedy algorithms
• When and why PINNs fail to train: A neural tangent kernel perspective
• The Finite Neuron Method and Convergence Analysis
• Convergence Rate Analysis for Deep Ritz Method
• Error estimates of residual minimization using neural networks for linear PDEs
• A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations

Cited by in corpus (6)

• A priori and a posteriori error estimates for the Deep Ritz method applied to the Laplace and Stokes problem
• Imaging Conductivity from Current Density Magnitude using Neural Networks
• On the Representation of Solutions to Elliptic PDEs in Barron Spaces
• Solving PDEs on Unknown Manifolds with Machine Learning
• Certified machine learning: Rigorous a posteriori error bounds for PDE defined PINNs
• Deep Ritz-Finite Element methods: Neural Network Methods trained with Finite Elements