Neural Operator: Graph Kernel Network for Partial Differential Equations

Abstract

The classical development of neural networks has been primarily for mappings between a finite-dimensional Euclidean space and a set of classes, or between two finite-dimensional Euclidean spaces. The purpose of this work is to generalize neural networks so that they can learn mappings between infinite-dimensional spaces (operators). The key innovation in our work is that a single set of network parameters, within a carefully designed network architecture, may be used to describe mappings between infinite-dimensional spaces and between different finite-dimensional approximations of those spaces. We formulate approximation of the infinite-dimensional mapping by composing nonlinear activation functions and a class of integral operators. The kernel integration is computed by message passing on graph networks. This approach has substantial practical consequences which we will illustrate in the context of mappings between input data to partial differential equations (PDEs) and their solutions. In this context, such learned networks can generalize among different approximation methods for the PDE (such as finite difference or finite element methods) and among approximations corresponding to different underlying levels of resolution and discretization. Experiments confirm that the proposed graph kernel network does have the desired properties and show competitive performance compared to the state of the art solvers.

References in corpus (10)

• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals
• Fast Graph Representation Learning with PyTorch Geometric
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• Deep Convolutional Networks as shallow Gaussian Processes
• Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs
• Unsupervised Deep Learning Algorithm for PDE-based Forward and Inverse Problems
• Deep Function Machines: Generalized Neural Networks for Topological Layer Expression

Cited by in corpus (45)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Fourier Neural Operator for Parametric Partial Differential Equations
• Machine learning accelerated computational fluid dynamics
• Physics-informed learning of governing equations from scarce data
• Enhancing Computational Fluid Dynamics with Machine Learning
• PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs
• EikoNet: Solving the Eikonal equation with Deep Neural Networks
• Multipole Graph Neural Operator for Parametric Partial Differential Equations
• Physics-Informed Neural Operator for Learning Partial Differential Equations
• Wavefield reconstruction inversion via physics-informed neural networks
• The Random Feature Model for Input-Output Maps between Banach Spaces
• Deep transfer operator learning for partial differential equations under conditional shift
• Long-term predictions of turbulence by implicit U-Net enhanced Fourier neural operator
• Multiwavelet-based Operator Learning for Differential Equations
• Attention-Enhanced Neural Network Models for Turbulence Simulation
• Data-driven discovery of Green's functions with human-understandable deep learning
• Mosaic Flows: A Transferable Deep Learning Framework for Solving PDEs on Unseen Domains
• Learning nonlocal constitutive models with neural networks
• Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• BubbleNet: Inferring micro-bubble dynamics with semi-physics-informed deep learning
• MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs
• Fully probabilistic deep models for forward and inverse problems in parametric PDEs
• A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
• Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers
• Neural Integral Equations
• PCNN: A physics-constrained neural network for multiphase flows
• Variational operator learning: A unified paradigm marrying training neural operators and solving partial differential equations
• Solving PDE-constrained Control Problems Using Operator Learning
• Deep Learning of Conjugate Mappings
• Accelerating Bayesian microseismic event location with deep learning
• Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods
• Structure Preservation for the Deep Neural Network Multigrid Solver
• Learning the structure of wind: A data-driven nonlocal turbulence model for the atmospheric boundary layer
• Learning phase field mean curvature flows with neural networks
• Parametric Complexity Bounds for Approximating PDEs with Neural Networks
• Frame-independent vector-cloud neural network for nonlocal constitutive modeling on arbitrary grids
• Inverting the Kohn-Sham equations with physics-informed machine learning
• Seismic wave propagation and inversion with Neural Operators
• Physics-informed Partitioned Coupled Neural Operator for Complex Networks
• DynaBench: A benchmark dataset for learning dynamical systems from low-resolution data
• Deep neural networks for geometric multigrid methods
• Banach neural operator for Navier-Stokes equations
• Learning Partial Differential Equations in Reproducing Kernel Hilbert Spaces
• Partial differential equation solver based on optimization methods
• Machine-learning custom-made basis functions for partial differential equations