Solving PDEs on Unknown Manifolds with Machine Learning
arXiv:2106.06682
Abstract
This paper proposes a mesh-free computational framework and machine learning theory for solving elliptic PDEs on unknown manifolds, identified with point clouds, based on diffusion maps (DM) and deep learning. The PDE solver is formulated as a supervised learning task to solve a least-squares regression problem that imposes an algebraic equation approximating a PDE (and boundary conditions if applicable). This algebraic equation involves a graph-Laplacian type matrix obtained via DM asymptotic expansion, which is a consistent estimator of second-order elliptic differential operators. The resulting numerical method is to solve a highly non-convex empirical risk minimization problem subjected to a solution from a hypothesis space of neural networks (NNs). In a well-posed elliptic PDE setting, when the hypothesis space consists of neural networks with either infinite width or depth, we show that the global minimizer of the empirical loss function is a consistent solution in the limit of large training data. When the hypothesis space is a two-layer neural network, we show that for a sufficiently large width, gradient descent can identify a global minimizer of the empirical loss function. Supporting numerical examples demonstrate the convergence of the solutions, ranging from simple manifolds with low and high co-dimensions, to rough surfaces with and without boundaries. We also show that the proposed NN solver can robustly generalize the PDE solution on new data points with generalization errors that are almost identical to the training errors, superseding a Nystrom-based interpolation method.
References in corpus (35)
- Adam: A Method for Stochastic Optimization
- DGM: A deep learning algorithm for solving partial differential equations
- Solving high-dimensional partial differential equations using deep learning
- Neural Tangent Kernel: Convergence and Generalization in Neural Networks
- Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
- A Convergence Theory for Deep Learning via Over-Parameterization
- A unified deep artificial neural network approach to partial differential equations in complex geometries
- A Mean Field View of the Landscape of Two-Layers Neural Networks
- Weak Adversarial Networks for High-dimensional Partial Differential Equations
- Gradient Descent Provably Optimizes Over-parameterized Neural Networks
- Deep Network Approximation for Smooth Functions
- Gradient Descent Finds Global Minima of Deep Neural Networks
- Convergence of the Deep BSDE Method for Coupled FBSDEs
- Optimal approximation of continuous functions by very deep ReLU networks
- D3M: A deep domain decomposition method for partial differential equations
- Deep splitting method for parabolic PDEs
- Neural Network Approximation: Three Hidden Layers Are Enough
- A Priori Estimates of the Population Risk for Two-layer Neural Networks
- Nonlinear Approximation and (Deep) ReLU Networks
- Optimal Approximation Rate of ReLU Networks in terms of Width and Depth
- Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques
- A finite area scheme for shallow granular flows on three-dimensional surfaces
- Deep Network with Approximation Error Being Reciprocal of Width to Power of Square Root of Depth
- The phase diagram of approximation rates for deep neural networks
- Convergence Rate Analysis for Deep Ritz Method
- Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory
- Overcoming the curse of dimensionality in the approximative pricing of financial derivatives with default risks
- A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations
- Approximating solutions of linear elliptic PDE's on a smooth manifold using local kernels
- Approximation in shift-invariant spaces with deep ReLU neural networks
- Spectral Convergence of Graph Laplacian and Heat Kernel Reconstruction in from Random Samples
- A Priori Analysis of Stable Neural Network Solutions to Numerical PDEs
- Learning Algebraic Multigrid Using Graph Neural Networks
- Reproducing Activation Function for Deep Learning
- The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation