Fourier Neural Operator for Parametric Partial Differential Equations

Abstract

The classical development of neural networks has primarily focused on learning mappings between finite-dimensional Euclidean spaces. Recently, this has been generalized to neural operators that learn mappings between function spaces. For partial differential equations (PDEs), neural operators directly learn the mapping from any functional parametric dependence to the solution. Thus, they learn an entire family of PDEs, in contrast to classical methods which solve one instance of the equation. In this work, we formulate a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture. We perform experiments on Burgers' equation, Darcy flow, and Navier-Stokes equation. The Fourier neural operator is the first ML-based method to successfully model turbulent flows with zero-shot super-resolution. It is up to three orders of magnitude faster compared to traditional PDE solvers. Additionally, it achieves superior accuracy compared to previous learning-based solvers under fixed resolution.

References in corpus (12)

• Machine learning accelerated computational fluid dynamics
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• Fast Training of Convolutional Networks through FFTs
• Implicit Neural Representations with Periodic Activation Functions
• EikoNet: Solving the Eikonal equation with Deep Neural Networks
• Neural Operator: Graph Kernel Network for Partial Differential Equations
• Physics-Informed Probabilistic Learning of Linear Embeddings of Non-linear Dynamics With Guaranteed Stability
• The Random Feature Model for Input-Output Maps between Banach Spaces
• A physics-informed operator regression framework for extracting data-driven continuum models
• BCR-Net: a neural network based on the nonstandard wavelet form
• Learning to Optimize Multigrid PDE Solvers
• Unsupervised Deep Learning Algorithm for PDE-based Forward and Inverse Problems

Cited by in corpus (187)

• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Enhancing Computational Fluid Dynamics with Machine Learning
• Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs
• Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios
• Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
• Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks
• Solving nonlinear differential equations with differentiable quantum circuits
• Physics-Informed Neural Operator for Learning Partial Differential Equations
• Real-time high-resolution CO${}_2$ geological storage prediction using nested Fourier neural operators
• Recent Advances and Applications of Machine Learning in Experimental Solid Mechanics: A Review
• Physics-Informed Neural Nets for Control of Dynamical Systems
• Wavefield reconstruction inversion via physics-informed neural networks
• The Random Feature Model for Input-Output Maps between Banach Spaces
• Deep transfer operator learning for partial differential equations under conditional shift
• Hybrid FEM-NN models: Combining artificial neural networks with the finite element method
• Reliable extrapolation of deep neural operators informed by physics or sparse observations
• Physics-informed neural networks of the Saint-Venant equations for downscaling a large-scale river model
• Interfacing Finite Elements with Deep Neural Operators for Fast Multiscale Modeling of Mechanics Problems
• Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization
• Self-supervised learning of hologram reconstruction using physics consistency
• Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations
• A Critical Review of Physics-Informed Machine Learning Applications in Subsurface Energy Systems
• An overview on deep learning-based approximation methods for partial differential equations
• Long-term predictions of turbulence by implicit U-Net enhanced Fourier neural operator
• Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics
• Physics-informed neural network methods based on Miura transformations and discovery of new localized wave solutions
• Evolutional Deep Neural Network
• Latent Space Data Assimilation by using Deep Learning
• Multiwavelet-based Operator Learning for Differential Equations
• Attention-Enhanced Neural Network Models for Turbulence Simulation
• Feasibility Study of Neural ODE and DAE Modules for Power System Dynamic Component Modeling
• Multi-fidelity prediction of fluid flow and temperature field based on transfer learning using Fourier Neural Operator
• Model-Parallel Fourier Neural Operators as Learned Surrogates for Large-Scale Parametric PDEs
• Data-driven discovery of Green's functions with human-understandable deep learning
• Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems
• Mosaic Flows: A Transferable Deep Learning Framework for Solving PDEs on Unseen Domains
• In-Context Operator Learning with Data Prompts for Differential Equation Problems
• Simulating progressive intramural damage leading to aortic dissection using an operator-regression neural network
• Learning macroscopic internal variables and history dependence from microscopic models
• SVD Perspectives for Augmenting DeepONet Flexibility and Interpretability
• Learning nonlocal constitutive models with neural networks
• Applications of physics informed neural operators
• A neural operator-based surrogate solver for free-form electromagnetic inverse design
• Improved generalization with deep neural operators for engineering systems: Path towards digital twin
• Nonlocal Kernel Network (NKN): a Stable and Resolution-Independent Deep Neural Network
• Rapid Seismic Waveform Modeling and Inversion with Neural Operators
• Physics-enhanced deep surrogates for partial differential equations
• Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors
• GaborPINN: Efficient physics informed neural networks using multiplicative filtered networks
• Plasma Surrogate Modelling using Fourier Neural Operators
• Complete quantum-inspired framework for computational fluid dynamics
• Learning elliptic partial differential equations with randomized linear algebra
• Hybrid quantum physics-informed neural networks for simulating computational fluid dynamics in complex shapes
• What machine learning can do for computational solid mechanics
• On the Use of Neural Networks for Full Waveform Inversion
• On universal approximation and error bounds for Fourier Neural Operators
• MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs
• BubbleNet: Inferring micro-bubble dynamics with semi-physics-informed deep learning
• Applications of Scientific Machine Learning for the Analysis of Functionally Graded Porous Beams
• Time Series Predictions in Unmonitored Sites: A Survey of Machine Learning Techniques in Water Resources
• Concurrent multi-parameter learning demonstrated on the Kuramoto-Sivashinsky equation
• Parallel Hybrid Networks: an interplay between quantum and classical neural networks
• A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
• Adaptive physics-informed neural operator for coarse-grained non-equilibrium flows
• Accelerating physics-informed neural network based 1D arc simulation by meta learning
• Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers
• Inferring electrochemical performance and parameters of Li-ion batteries based on deep operator networks
• Fast Dynamic 1D Simulation of Divertor Plasmas with Neural PDE Surrogates
• Finite element based model order reduction for parametrized one-way coupled steady state linear thermomechanical problems
• Neural Integral Equations
• Learning Physical Models that Can Respect Conservation Laws
• Magnetohydrodynamics with Physics Informed Neural Operators
• Deep Learning to Estimate Permeability using Geophysical Data
• Deep learning fluid flow reconstruction around arbitrary two-dimensional objects from sparse sensors using conformal mappings
• Towards Optimally Weighted Physics-Informed Neural Networks in Ocean Modelling
• PCNN: A physics-constrained neural network for multiphase flows
• Generative Learning of the Solution of Parametric Partial Differential Equations Using Guided Diffusion Models and Virtual Observations
• Variational operator learning: A unified paradigm marrying training neural operators and solving partial differential equations
• A Multifidelity deep operator network approach to closure for multiscale systems
• Accelerating Phase Field Simulations Through a Hybrid Adaptive Fourier Neural Operator with U-Net Backbone
• Neural Cellular Automata for Solidification Microstructure Modelling
• Learned multiphysics inversion with differentiable programming and machine learning
• Machine learning surrogates for efficient hydrologic modeling: Insights from stochastic simulations of managed aquifer recharge
• Comparison of Neural FEM and Neural Operator Methods for applications in Solid Mechanics
• Meta-Auto-Decoder for Solving Parametric Partial Differential Equations
• Elliptic PDE learning is provably data-efficient
• Solving PDE-constrained Control Problems Using Operator Learning
• Machine learning for phase-resolved reconstruction of nonlinear ocean wave surface elevations from sparse remote sensing data
• Multiscale DeepONet for Nonlinear Operators in Oscillatory Function Spaces for Building Seismic Wave Responses
• Multi-scale Time-stepping of Partial Differential Equations with Transformers
• Machine-learning heat flux closure for multi-moment fluid modeling of nonlinear Landau damping
• Machine-Learning-Assisted Photonic Device Development: A Multiscale Approach from Theory to Characterization
• Beltrami Flow and Neural Diffusion on Graphs
• Deep Learning of Conjugate Mappings
• Operator Learning with Neural Fields: Tackling PDEs on General Geometries
• Spectrally Adapted Physics-Informed Neural Networks for Solving Unbounded Domain Problems
• Continuous PDE Dynamics Forecasting with Implicit Neural Representations
• Elvet -- a neural network-based differential equation and variational problem solver
• Predicting atmospheric turbulence for secure quantum communications in free space
• Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings
• Modelling parametric uncertainty in PDEs models via Physics-Informed Neural Networks
• Deep learning for nano-photonic materials -- The solution to everything!?
• An Advanced Physics-Informed Neural Operator for Comprehensive Design Optimization of Highly-Nonlinear Systems: An Aerospace Composites Processing Case Study
• Neural Ordinary Differential Equations for Model Order Reduction of Stiff Systems
• Convergence rate of DeepONets for learning operators arising from advection-diffusion equations
• LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data
• Gradient and Uncertainty Enhanced Sequential Sampling for Global Fit
• Solving Partial Differential Equations with Point Source Based on Physics-Informed Neural Networks
• Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimality
• Differentiable Physics: A Position Piece
• Physics-Aware Neural Implicit Solvers for multiscale, parametric PDEs with applications in heterogeneous media
• Physics-informed Convolutional Neural Networks for Temperature Field Prediction of Heat Source Layout without Labeled Data
• Structure Preservation for the Deep Neural Network Multigrid Solver
• Finite-difference-informed graph network for solving steady-state incompressible flows on block-structured grids
• Learning the structure of wind: A data-driven nonlocal turbulence model for the atmospheric boundary layer
• Can we integrate spatial verification methods into neural-network loss functions for atmospheric science?
• Multi-Fidelity Machine Learning Applied to Steady Fluid Flows
• Physics Encoded Blocks in Residual Neural Network Architectures for Digital Twin Models
• Image-Based Soil Organic Carbon Remote Sensing from Satellite Images with Fourier Neural Operator and Structural Similarity
• On the Representation of Solutions to Elliptic PDEs in Barron Spaces
• Promoting global stability in data-driven models of quadratic nonlinear dynamics
• Enhancing generalization in endwall film cooling prediction: Incorporating the superposition principle into transformer-based neural operators
• Novel Deep neural networks for solving Bayesian statistical inverse
• Augmenting a Physics-Informed Neural Network for the 2D Burgers Equation by Addition of Solution Data Points
• Machine learning-based vorticity evolution and superresolution of homogeneous isotropic turbulence using wavelet projection
• Understanding complex crowd dynamics with generative neural simulators
• How more data can hurt: Instability and regularization in next-generation reservoir computing
• Stochastic Latent Transformer: Efficient Modelling of Stochastically Forced Zonal Jets
• Solving the Wide-band Inverse Scattering Problem via Equivariant Neural Networks
• Accelerating innovation with software abstractions for scalable computational geophysics
• Physics Informed Deep Learning for Flow and Transport in Porous Media
• Parametric Complexity Bounds for Approximating PDEs with Neural Networks
• Frame invariance and scalability of neural operators for partial differential equations
• Seismic wave propagation and inversion with Neural Operators
• Learning the dynamics of a one-dimensional plasma model with graph neural networks
• On Bayesian data assimilation for PDEs with ill-posed forward problems
• Variational Monte Carlo Approach to Partial Differential Equations with Neural Networks
• Composed Physics- and Data-driven System Identification for Non-autonomous Systems in Control Engineering
• Learning Density Distribution of Reachable States for Autonomous Systems
• FNOSeg3D: Resolution-Robust 3D Image Segmentation with Fourier Neural Operator
• One-shot learning for solution operators of partial differential equations
• Physics-Guided Discovery of Highly Nonlinear Parametric Partial Differential Equations
• Residual U-Net for accurate and efficient prediction of hemodynamics in two-dimensional asymmetric stenosis
• Physics-informed Partitioned Coupled Neural Operator for Complex Networks
• Efficient PINNs via Multi-Head Unimodular Regularization of the Solutions Space
• A composable autoencoder-based iterative algorithm for accelerating numerical simulations
• A Model-Constrained Tangent Slope Learning Approach for Dynamical Systems
• A reduced order Schwarz method for nonlinear multiscale elliptic equations based on two-layer neural networks
• Asymptotic-Preserving Neural Networks for Multiscale Time-Dependent Linear Transport Equations
• Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data
• Banach neural operator for Navier-Stokes equations
• Deep neural networks for geometric multigrid methods
• Going Deeper in Frequency Convolutional Neural Network: A Theoretical Perspective
• BINet: Learning to Solve Partial Differential Equations with Boundary Integral Networks
• Improving Deep Learning Performance for Predicting Large-Scale Porous-Media Flow through Feature Coarsening
• The Deep Parametric PDE Method: Application to Option Pricing
• Boundary-to-Solution Mapping for Groundwater Flows in a Toth Basin
• Evaluation of Deep Neural Operator Models toward Ocean Forecasting
• Physics Informed Neural Networks (PINNs)for approximating nonlinear dispersive PDEs
• HartleyMHA: Self-Attention in Frequency Domain for Resolution-Robust and Parameter-Efficient 3D Image Segmentation
• On the approximation of functions by tanh neural networks
• Learning Runge-Kutta Integration Schemes for ODE Simulation and Identification
• Quantitative analysis of the prediction performance of a Convolutional Neural Network evaluating the surface elastic energy of a strained film
• Learning Partial Differential Equations in Reproducing Kernel Hilbert Spaces
• Stability of implicit neural networks for long-term forecasting in dynamical systems
• Semi-Implicit Neural Solver for Time-dependent Partial Differential Equations
• Solving an elastic inverse problem using Convolutional Neural Networks
• Surrogate-data-enriched Physics-Aware Neural Networks
• Approximation of Functionals by Neural Network without Curse of Dimensionality
• Discovering Hidden Physics Behind Transport Dynamics
• Superstep wavefield propagation
• Evolving Generalizable Multigrid-Based Helmholtz Preconditioners with Grammar-Guided Genetic Programming
• Neural Fixed-Point Acceleration for Convex Optimization
• Efficient Simulation of Complex Capillary Effects in Advanced Manufacturing Processes using the Finite Volume Method
• Modeling turbulent and self-gravitating fluids with Fourier neural operators
• High Throughput Training of Deep Surrogates from Large Ensemble Runs
• The Pseudo Projection Operator: Applications of Deep Learning to Projection Based Filtering in Non-Trivial Frequency Regimes
• Solving time-dependent parametric PDEs by multiclass classification-based reduced order model
• Generative Network-Based Reduced-Order Model for Prediction, Data Assimilation and Uncertainty Quantification
• Orthogonal Transforms in Neural Networks Amount to Effective Regularization
• Partial differential equation solver based on optimization methods
• FC2T2: The Fast Continuous Convolutional Taylor Transform with Applications in Vision and Graphics
• Machine-learning custom-made basis functions for partial differential equations
• Algebraically-Informed Deep Networks (AIDN): A Deep Learning Approach to Represent Algebraic Structures
• How Analysis Can Teach Us the Optimal Way to Design Neural Operators