Tackling the Curse of Dimensionality with Physics-Informed Neural Networks

Abstract

The curse-of-dimensionality taxes computational resources heavily with exponentially increasing computational cost as the dimension increases. This poses great challenges in solving high-dimensional PDEs, as Richard E. Bellman first pointed out over 60 years ago. While there has been some recent success in solving numerically partial differential equations (PDEs) in high dimensions, such computations are prohibitively expensive, and true scaling of general nonlinear PDEs to high dimensions has never been achieved. We develop a new method of scaling up physics-informed neural networks (PINNs) to solve arbitrary high-dimensional PDEs. The new method, called Stochastic Dimension Gradient Descent (SDGD), decomposes a gradient of PDEs into pieces corresponding to different dimensions and randomly samples a subset of these dimensional pieces in each iteration of training PINNs. We prove theoretically the convergence and other desired properties of the proposed method. We demonstrate in various diverse tests that the proposed method can solve many notoriously hard high-dimensional PDEs, including the Hamilton-Jacobi-Bellman (HJB) and the Schrödinger equations in tens of thousands of dimensions very fast on a single GPU using the PINNs mesh-free approach. Notably, we solve nonlinear PDEs with nontrivial, anisotropic, and inseparable solutions in 100,000 effective dimensions in 12 hours on a single GPU using SDGD with PINNs. Since SDGD is a general training methodology of PINNs, it can be applied to any current and future variants of PINNs to scale them up for arbitrary high-dimensional PDEs.

Accepted by Neural Networks. Code is available at https://github.com/zheyuanhu01/SDGD_PINN

References in corpus (11)

• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology
• Hutchinson Trace Estimation for High-Dimensional and High-Order Physics-Informed Neural Networks
• On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems
• Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations
• A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations
• Separable PINN: Mitigating the Curse of Dimensionality in Physics-Informed Neural Networks
• Bias-Variance Trade-off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs
• Neural Decomposition: Functional ANOVA with Variational Autoencoders
• Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck Equations
• Deep neural operators can serve as accurate surrogates for shape optimization: A case study for airfoils

Cited by in corpus (12)

• Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?
• Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants
• Tackling the Curse of Dimensionality in Fractional and Tempered Fractional PDEs with Physics-Informed Neural Networks
• Learning thermoacoustic interactions in combustors using a physics-informed neural network
• A Practitioner's Guide to Kolmogorov-Arnold Networks
• Separable Physics-informed Neural Networks for Solving the BGK Model of the Boltzmann Equation
• An efficient wavelet-based physics-informed neural network for multiscale problems
• Physics-informed neural networks for solving functional renormalization group on a lattice
• Neural semi-Lagrangian method for high-dimensional advection-diffusion problems
• Lower Bound on the Representation Complexity of Antisymmetric Tensor Product Functions
• Quasi-Random Physics-informed Neural Networks
• A Variational Framework for the Complexity of PDE Solutions