On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton--Jacobi partial differential equations

Abstract

We propose novel connections between several neural network architectures and viscosity solutions of some Hamilton--Jacobi (HJ) partial differential equations (PDEs) whose Hamiltonian is convex and only depends on the spatial gradient of the solution. To be specific, we prove that under certain assumptions, the two neural network architectures we proposed represent viscosity solutions to two sets of HJ PDEs with zero error. We also implement our proposed neural network architectures using Tensorflow and provide several examples and illustrations. Note that these neural network representations can avoid curve of dimensionality for certain HJ PDEs, since they do not involve neither grids nor discretization. Our results suggest that efficient dedicated hardware implementation for neural networks can be leveraged to evaluate viscosity solutions of certain HJ PDEs.

References in corpus (39)

• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• fPINNs: Fractional Physics-Informed Neural Networks
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Solving Many-Electron Schrödinger Equation Using Deep Neural Networks
• PDE-Net: Learning PDEs from Data
• Deep learning observables in computational fluid dynamics
• Deep splitting method for parabolic PDEs
• The max-plus finite element method for solving deterministic optimal control problems: basic properties and convergence analysis
• Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis
• Deep Learning the Physics of Transport Phenomena
• Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs
• Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
• Curse of dimensionality reduction in max-plus based approximation methods: theoretical estimates and improved pruning algorithms
• Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton-Jacobi equations
• Learning Neural PDE Solvers with Convergence Guarantees
• Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations
• A Discussion on Solving Partial Differential Equations using Neural Networks
• Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems
• A new scalable algorithm for computational optimal control under uncertainty
• A deep learning approach for the computation of curvature in the level-set method
• On Decomposition Models in Imaging Sciences and Multi-time Hamilton-Jacobi Partial Differential Equations
• Using Neural Networks to Compute Approximate and Guaranteed Feasible Hamilton-Jacobi-Bellman PDE Solutions
• Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks
• Study on a Poisson's Equation Solver Based On Deep Learning Technique
• Solving Inverse Wave Scattering with Deep Learning
• An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems
• Recursive Regression with Neural Networks: Approximating the HJI PDE Solution
• K-TanH: Efficient TanH For Deep Learning

Cited by in corpus (13)

• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
• Adaptive Deep Learning for High-Dimensional Hamilton-Jacobi-Bellman Equations
• An overview on deep learning-based approximation methods for partial differential equations
• Neural Eikonal Solver: improving accuracy of physics-informed neural networks for solving eikonal equation in case of caustics
• Neural network architectures using min-plus algebra for solving certain high dimensional optimal control problems and Hamilton-Jacobi PDEs
• Physics Encoded Blocks in Residual Neural Network Architectures for Digital Twin Models
• An adaptive sparse grid local discontinuous Galerkin method for Hamilton-Jacobi equations in high dimensions
• Lax-Oleinik-type formulas and efficient algorithms for certain high-dimensional optimal control problems
• On representation formulas for optimal control: A Lagrangian perspective
• A Computationally Efficient Hamilton-Jacobi-based Formula for State-Constrained Optimal Control Problems
• Incorporating Coupling Knowledge into Echo State Networks for Learning Spatiotemporally Chaotic Dynamics
• Hopf-type representation formulas and efficient algorithms for certain high-dimensional optimal control problems