Solving and Learning Nonlinear PDEs with Gaussian Processes

Abstract

We introduce a simple, rigorous, and unified framework for solving nonlinear partial differential equations (PDEs), and for solving inverse problems (IPs) involving the identification of parameters in PDEs, using the framework of Gaussian processes. The proposed approach: (1) provides a natural generalization of collocation kernel methods to nonlinear PDEs and IPs; (2) has guaranteed convergence for a very general class of PDEs, and comes equipped with a path to compute error bounds for specific PDE approximations; (3) inherits the state-of-the-art computational complexity of linear solvers for dense kernel matrices. The main idea of our method is to approximate the solution of a given PDE as the maximum a posteriori (MAP) estimator of a Gaussian process conditioned on solving the PDE at a finite number of collocation points. Although this optimization problem is infinite-dimensional, it can be reduced to a finite-dimensional one by introducing additional variables corresponding to the values of the derivatives of the solution at collocation points; this generalizes the representer theorem arising in Gaussian process regression. The reduced optimization problem has the form of a quadratic objective function subject to nonlinear constraints; it is solved with a variant of the Gauss--Newton method. The resulting algorithm (a) can be interpreted as solving successive linearizations of the nonlinear PDE, and (b) in practice is found to converge in a small number of iterations (2 to 10), for a wide range of PDEs. Most traditional approaches to IPs interleave parameter updates with numerical solution of the PDE; our algorithm solves for both parameter and PDE solution simultaneously. Experiments on nonlinear elliptic PDEs, Burgers' equation, a regularized Eikonal equation, and an IP for permeability identification in Darcy flow illustrate the efficacy and scope of our framework.

arXiv comment41 pages

References in corpus (13)

• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Learning data driven discretizations for partial differential equations
• Multipole Graph Neural Operator for Parametric Partial Differential Equations
• MAP Estimators and Their Consistency in Bayesian Nonparametric Inverse Problems
• The Random Feature Model for Input-Output Maps between Banach Spaces
• When and why PINNs fail to train: A neural tangent kernel perspective
• Kernel Flows: from learning kernels from data into the abyss
• Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients
• Probabilistic ODE Solvers with Runge-Kutta Means

Cited by in corpus (3)

• Learning particle swarming models from data with Gaussian processes
• Machine learning of continuous and discrete variational ODEs with convergence guarantee and uncertainty quantification
• Learning Partial Differential Equations in Reproducing Kernel Hilbert Spaces