PDE-Net: Learning PDEs from Data

Abstract

In this paper, we present an initial attempt to learn evolution PDEs from data. Inspired by the latest development of neural network designs in deep learning, we propose a new feed-forward deep network, called PDE-Net, to fulfill two objectives at the same time: to accurately predict dynamics of complex systems and to uncover the underlying hidden PDE models. The basic idea of the proposed PDE-Net is to learn differential operators by learning convolution kernels (filters), and apply neural networks or other machine learning methods to approximate the unknown nonlinear responses. Comparing with existing approaches, which either assume the form of the nonlinear response is known or fix certain finite difference approximations of differential operators, our approach has the most flexibility by learning both differential operators and the nonlinear responses. A special feature of the proposed PDE-Net is that all filters are properly constrained, which enables us to easily identify the governing PDE models while still maintaining the expressive and predictive power of the network. These constrains are carefully designed by fully exploiting the relation between the orders of differential operators and the orders of sum rules of filters (an important concept originated from wavelet theory). We also discuss relations of the PDE-Net with some existing networks in computer vision such as Network-In-Network (NIN) and Residual Neural Network (ResNet). Numerical experiments show that the PDE-Net has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.

15 pages, 13 figures

Cited by in corpus (86)

• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• Efficient training of physics-informed neural networks via importance sampling
• Stiff-PINN: Physics-Informed Neural Network for Stiff Chemical Kinetics
• Data Driven Governing Equations Approximation Using Deep Neural Networks
• Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Network
• Data-Driven Deep Learning of Partial Differential Equations in Modal Space
• Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method
• Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations
• Deep splitting method for parabolic PDEs
• D3M: A deep domain decomposition method for partial differential equations
• The Random Feature Model for Input-Output Maps between Banach Spaces
• DeepMoD: Deep learning for Model Discovery in noisy data
• Hybrid FEM-NN models: Combining artificial neural networks with the finite element method
• MgNet: A Unified Framework of Multigrid and Convolutional Neural Network
• Solving Electrical Impedance Tomography with Deep Learning
• Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting
• SPINN: Sparse, Physics-based, and partially Interpretable Neural Networks for PDEs
• Uniformly Accurate Machine Learning Based Hydrodynamic Models for Kinetic Equations
• An overview on deep learning-based approximation methods for partial differential equations
• Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)
• 2022 Review of Data-Driven Plasma Science
• FEA-Net: A Physics-guided Data-driven Model for Efficient Mechanical Response Prediction
• Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning
• Solving inverse-PDE problems with physics-aware neural networks
• Variational Onsager Neural Networks (VONNs): A thermodynamics-based variational learning strategy for non-equilibrium PDEs
• Reversible Architectures for Arbitrarily Deep Residual Neural Networks
• On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton--Jacobi partial differential equations
• A Multi-Pass GAN for Fluid Flow Super-Resolution
• PDE-based Group Equivariant Convolutional Neural Networks
• Poisson CNN: Convolutional neural networks for the solution of the Poisson equation on a Cartesian mesh
• Compressed Convolutional LSTM: An Efficient Deep Learning framework to Model High Fidelity 3D Turbulence
• On generalized residue network for deep learning of unknown dynamical systems
• Physics-Informed Supervised Residual Learning for Electromagnetic Modeling
• Learning Dynamical Systems from Partial Observations
• Learning nonlocal constitutive models with neural networks
• Embedding Hard Physical Constraints in Neural Network Coarse-Graining of 3D Turbulence
• Overcoming the curse of dimensionality in the numerical approximation of Allen-Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations
• Learning interaction kernels in heterogeneous systems of agents from multiple trajectories
• A Helmholtz equation solver using unsupervised learning: Application to transcranial ultrasound
• Variational training of neural network approximations of solution maps for physical models
• Reducing the complexity of chemical networks via interpretable autoencoders
• Numerical simulations for full history recursive multilevel Picard approximations for systems of high-dimensional partial differential equations
• Online learning of both state and dynamics using ensemble Kalman filters
• PySINDy: A Python package for the Sparse Identification of Nonlinear Dynamics from Data
• Physics-driven Learning of the Steady Navier-Stokes Equations using Deep Convolutional Neural Networks
• Blending Diverse Physical Priors with Neural Networks
• Data-driven discovery of free-form governing differential equations
• Sparse Identification of Truncation Errors
• VarNet: Variational Neural Networks for the Solution of Partial Differential Equations
• FiniteNet: A Fully Convolutional LSTM Network Architecture for Time-Dependent Partial Differential Equations
• Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations
• A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables
• NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data
• IDENT: Identifying Differential Equations with Numerical Time evolution
• Data-driven, multi-moment fluid modeling of Landau damping
• Deep neural network approximations for Monte Carlo algorithms
• Spectrally Adapted Physics-Informed Neural Networks for Solving Unbounded Domain Problems
• Stability selection enables robust learning of partial differential equations from limited noisy data
• Operator Learning with Neural Fields: Tackling PDEs on General Geometries
• Learning orbital dynamics of binary black hole systems from gravitational wave measurements
• KiNet: A Deep Neural Network Representation of Chemical Kinetics
• Data-driven Discovery of Partial Differential Equations for Multiple-Physics Electromagnetic Problem
• Deep Diffeomorphic Normalizing Flows
• SwitchNet: a neural network model for forward and inverse scattering problems
• MeshingNet: A New Mesh Generation Method based on Deep Learning
• Learning continuous-time PDEs from sparse data with graph neural networks
• Extracting structured dynamical systems using sparse optimization with very few samples
• Overcoming the curse of dimensionality for some Hamilton--Jacobi partial differential equations via neural network architectures
• Discretization of parameter identification in PDEs using Neural Networks
• FD-Net with Auxiliary Time Steps: Fast Prediction of PDEs using Hessian-Free Trust-Region Methods
• Data-driven learning of non-autonomous systems
• Forward and backward problems for coupled subdiffusion systems
• A Deep Learning Approach for Predicting Spatiotemporal Dynamics From Sparsely Observed Data
• Neural ODEs for Image Segmentation with Level Sets
• A Neural Solver for Variational Problems on CAD Geometries with Application to Electric Machine Simulation
• A reduced order Schwarz method for nonlinear multiscale elliptic equations based on two-layer neural networks
• TIME: A Transparent, Interpretable, Model-Adaptive and Explainable Neural Network for Dynamic Physical Processes
• Differential equations as models of deep neural networks
• Deep-learning PDEs with unlabeled data and hardwiring physics laws
• Assessment of End-to-End and Sequential Data-driven Learning of Fluid Flows
• ${\ell }_1$DecNet+: A new architecture framework by ${\ell }_1$ decomposition and iteration unfolding for sparse feature segmentation
• LMKL-Net: A Fast Localized Multiple Kernel Learning Solver via Deep Neural Networks
• Learning the Dynamics of Sparsely Observed Interacting Systems
• A Grid-Structured Model of Tubular Reactors