Convergence Rate Analysis for Deep Ritz Method

Abstract

Using deep neural networks to solve PDEs has attracted a lot of attentions recently. However, why the deep learning method works is falling far behind its empirical success. In this paper, we provide a rigorous numerical analysis on deep Ritz method (DRM) \cite{wan11} for second order elliptic equations with Neumann boundary conditions. We establish the first nonasymptotic convergence rate in $H^1$ norm for DRM using deep networks with ${\mathrm{ReLU}}^2$ activation functions. In addition to providing a theoretical justification of DRM, our study also shed light on how to set the hyper-parameter of depth and width to achieve the desired convergence rate in terms of number of training samples. Technically, we derive bounds on the approximation error of deep ${\mathrm{ReLU}}^2$ network in $H^1$ norm and on the Rademacher complexity of the non-Lipschitz composition of gradient norm and ${\mathrm{ReLU}}^2$ network, both of which are of independent interest.

References in corpus (11)

• DGM: A deep learning algorithm for solving partial differential equations
• fPINNs: Fractional Physics-Informed Neural Networks
• Weak Adversarial Networks for High-dimensional Partial Differential Equations
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality
• The Finite Neuron Method and Convergence Analysis
• Deep ReLU network approximation of functions on a manifold
• Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory
• Error estimates of residual minimization using neural networks for linear PDEs
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs
• A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations

Cited by in corpus (7)

• A rate of convergence of Physics Informed Neural Networks for the linear second order elliptic PDEs
• Deep unfitted Nitsche method for elliptic interface problems
• Imaging Conductivity from Current Density Magnitude using Neural Networks
• Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimality
• A Priori Analysis of Stable Neural Network Solutions to Numerical PDEs
• Solving PDEs on Unknown Manifolds with Machine Learning
• Stationary Density Estimation of Itô Diffusions Using Deep Learning