A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data

Abstract

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019, and the second one, named Fourier Neural Operator or FNO, was published in 2020. In order to compare FNO with DeepONet for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also endow DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO. We consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries and especially noisy data, the performance of FNO deteriorates greatly. For example, for the instability wave analysis with only 0.1% noise added to the input data, the error of FNO increases 10000 times making it inappropriate for such important applications, while there is hardly any effect of such noise on the DeepONet. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

References in corpus (28)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• DeepXDE: A deep learning library for solving differential equations
• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• Fourier Neural Operator for Parametric Partial Differential Equations
• fPINNs: Fractional Physics-Informed Neural Networks
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
• Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• A physics-informed variational DeepONet for predicting the crack path in brittle materials
• The Unreasonable Effectiveness of Deep Learning in Artificial Intelligence
• Collapse of Deep and Narrow Neural Nets
• Dying ReLU and Initialization: Theory and Numerical Examples
• DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks
• Multi-fidelity Bayesian Neural Networks: Algorithms and Applications
• A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks
• Neural Operator: Graph Kernel Network for Partial Differential Equations
• Operator learning for predicting multiscale bubble growth dynamics
• A physics-informed operator regression framework for extracting data-driven continuum models
• Data-driven learning of robust nonlocal physics from high-fidelity synthetic data
• Physics-informed neural networks (PINNs) for fluid mechanics: A review
• Learning Functional Priors and Posteriors from Data and Physics
• On universal approximation and error bounds for Fourier Neural Operators
• DeepONet prediction of linear instability waves in high-speed boundary layers
• DeepM&Mnet for hypersonics: Predicting the coupled flow and finite-rate chemistry behind a normal shock using neural-network approximation of operators
• Convergence rate of DeepONets for learning operators arising from advection-diffusion equations
• Arbitrary-Depth Universal Approximation Theorems for Operator Neural Networks