Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations

Abstract

While there is currently a lot of enthusiasm about "big data", useful data is usually "small" and expensive to acquire. In this paper, we present a new paradigm of learning partial differential equations from {\em small} data. In particular, we introduce \emph{hidden physics models}, which are essentially data-efficient learning machines capable of leveraging the underlying laws of physics, expressed by time dependent and nonlinear partial differential equations, to extract patterns from high-dimensional data generated from experiments. The proposed methodology may be applied to the problem of learning, system identification, or data-driven discovery of partial differential equations. Our framework relies on Gaussian processes, a powerful tool for probabilistic inference over functions, that enables us to strike a balance between model complexity and data fitting. The effectiveness of the proposed approach is demonstrated through a variety of canonical problems, spanning a number of scientific domains, including the Navier-Stokes, Schrödinger, Kuramoto-Sivashinsky, and time dependent linear fractional equations. The methodology provides a promising new direction for harnessing the long-standing developments of classical methods in applied mathematics and mathematical physics to design learning machines with the ability to operate in complex domains without requiring large quantities of data.

References in corpus (15)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Applied Koopmanism
• Chaos as an Intermittently Forced Linear System
• Machine Learning of Linear Differential Equations using Gaussian Processes
• Gaussian Processes for Big Data
• Structure Discovery in Nonparametric Regression through Compositional Kernel Search
• Predicting catastrophes in nonlinear dynamical systems by compressive sensing
• Inferring solutions of differential equations using noisy multi-fidelity data
• Compressed Modes for Variational Problems in Mathematics and Physics
• Sparse Dynamics for Partial Differential Equations
• Compressive sensing and low-rank libraries for classification of bifurcation regimes in nonlinear dynamical systems
• Efficient inference of parsimonious phenomenological models of cellular dynamics using S-systems and alternating regression
• Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (2013)
• Deep Multi-fidelity Gaussian Processes
• Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations

Cited by in corpus (276)

• Machine Learning for Fluid Mechanics
• An overview of deep learning in medical imaging focusing on MRI
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• fPINNs: Fractional Physics-Informed Neural Networks
• Explainable Machine Learning for Scientific Insights and Discoveries
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Data-driven discovery of coordinates and governing equations
• hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition
• A review of machine learning applications in wildfire science and management
• Integrating Machine Learning and Multiscale Modeling: Perspectives, Challenges, and Opportunities in the Biological, Biomedical, and Behavioral Sciences
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networks
• Neural Ordinary Differential Equations
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Discovering physical concepts with neural networks
• A Deep Collocation Method for the Bending Analysis of Kirchhoff Plate
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• The Challenge of Machine Learning in Space Weather Nowcasting and Forecasting
• Deep Learning of Vortex Induced Vibrations
• Sub-grid modelling for two-dimensional turbulence using neural networks
• Data Driven Governing Equations Approximation Using Deep Neural Networks
• SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• Shallow Neural Networks for Fluid Flow Reconstruction with Limited Sensors
• A Physics-Informed Deep Learning Paradigm for Car-Following Models
• Variational Physics-Informed Neural Networks For Solving Partial Differential Equations
• Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
• A neural network approach for the blind deconvolution of turbulent flows
• A two-stage physics-informed neural network method based on conserved quantities and applications in localized wave solutions
• Data-Driven Deep Learning of Partial Differential Equations in Modal Space
• Deep learning observables in computational fluid dynamics
• Tackling the Curse of Dimensionality with Physics-Informed Neural Networks
• Physics Informed Neural Networks for Simulating Radiative Transfer
• Deep Learning in Deterministic Computational Mechanics
• Data-driven discovery of PDEs in complex datasets
• Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology
• Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems
• Data-driven rogue waves and parameter discovery in the defocusing NLS equation with a potential using the PINN deep learning
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• A Perspective on Machine Learning Methods in Turbulence Modelling
• Physics-informed neural networks for solving forward and inverse problems in complex beam systems
• Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers
• Using Noisy or Incomplete Data to Discover Models of Spatiotemporal Dynamics
• A Physics-Informed Deep Learning Paradigm for Traffic State and Fundamental Diagram Estimation
• Distributed physics informed neural network for data-efficient solution to partial differential equations
• On Learning Hamiltonian Systems from Data
• Solving Electrical Impedance Tomography with Deep Learning
• Pricing options and computing implied volatilities using neural networks
• A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge
• Neural Jump Stochastic Differential Equations
• Stochastic analysis of heterogeneous porous material with modified neural architecture search (NAS) based physics-informed neural networks using transfer learning
• Data-driven recovery of hidden physics in reduced order modeling of fluid flows
• Physics-constrained, low-dimensional models for MHD: First-principles and data-driven approaches
• An Efficient Deep Learning Technique for the Navier-Stokes Equations: Application to Unsteady Wake Flow Dynamics
• DynNet: Physics-based neural architecture design for linear and nonlinear structural response modeling and prediction
• Uniformly Accurate Machine Learning Based Hydrodynamic Models for Kinetic Equations
• Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis
• Autonomous inference of complex network dynamics from incomplete and noisy data
• A Physics-Constrained Data-Driven Approach Based on Locally Convex Reconstruction for Noisy Database
• Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks
• 2022 Review of Data-Driven Plasma Science
• Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data
• Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning
• Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation
• Variational system identification of the partial differential equations governing pattern-forming physics: Inference under varying fidelity and noise
• Classification of chaotic time series with deep learning
• Solving inverse-PDE problems with physics-aware neural networks
• Variational Onsager Neural Networks (VONNs): A thermodynamics-based variational learning strategy for non-equilibrium PDEs
• Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensors
• Coarse-scale PDEs from fine-scale observations via machine learning
• Learned discretizations for passive scalar advection in a 2-D turbulent flow
• Weak Form Theory-guided Neural Network (TgNN-wf) for Deep Learning of Subsurface Single and Two-phase Flow
• Active Training of Physics-Informed Neural Networks to Aggregate and Interpolate Parametric Solutions to the Navier-Stokes Equations
• Physics-Informed Deep Learning For Traffic State Estimation: A Survey and the Outlook
• Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows
• A Derivative-Free Method for Solving Elliptic Partial Differential Equations with Deep Neural Networks
• Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models
• Data-driven peakon and periodic peakon travelling wave solutions of some nonlinear dispersive equations via deep learning
• Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
• Physics-informed deep generative models
• On generalized residue network for deep learning of unknown dynamical systems
• Thermodynamics-informed graph neural networks
• DySMHO: Data-Driven Discovery of Governing Equations for Dynamical Systems via Moving Horizon Optimization
• Using Deep Learning to Extend the Range of Air-Pollution Monitoring and Forecasting
• Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations
• BCR-Net: a neural network based on the nonstandard wavelet form
• Symplectic Neural Networks in Taylor Series Form for Hamiltonian Systems
• A neural network multigrid solver for the Navier-Stokes equations
• Constructing coarse-scale bifurcation diagrams from spatio-temporal observations of microscopic simulations: A parsimonious machine learning approach
• Mesoscopic simulations at the physics-chemistry-biology interface
• A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence
• Deep learning neural networks for the third-order nonlinear Schrodinger equation: Solitons, breathers, and rogue waves
• Differentiable Physics-informed Graph Networks
• Neural Network Forecast of the Sunspot Butterfly Diagram
• Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors
• Learning interaction kernels in heterogeneous systems of agents from multiple trajectories
• A physics-aware, probabilistic machine learning framework for coarse-graining high-dimensional systems in the Small Data regime
• Linking Machine Learning with Multiscale Numerics: Data-Driven Discovery of Homogenized Equations
• Probabilistic learning on manifolds constrained by nonlinear partial differential equations for small datasets
• Learning elliptic partial differential equations with randomized linear algebra
• Data-aided Underwater Acoustic Ray Propagation Modeling
• Coercing Machine Learning to Output Physically Accurate Results
• Deep-learning based discovery of partial differential equations in integral form from sparse and noisy data
• Reducing the complexity of chemical networks via interpretable autoencoders
• Identification of physical processes via combined data-driven and data-assimilation methods
• Error estimates of residual minimization using neural networks for linear PDEs
• On universal approximation and error bounds for Fourier Neural Operators
• Energy-based error bound of physics-informed neural network solutions in elasticity
• PySINDy: A Python package for the Sparse Identification of Nonlinear Dynamics from Data
• Physics-driven Learning of the Steady Navier-Stokes Equations using Deep Convolutional Neural Networks
• Data-driven discovery of free-form governing differential equations
• Fully probabilistic deep models for forward and inverse problems in parametric PDEs
• Bayesian deep learning with hierarchical prior: Predictions from limited and noisy data
• Data-driven discoveries of Bäcklund transforms and soliton evolution equations via deep neural network learning schemes
• Neural Network Training Using ${\ell }_1$-Regularization and Bi-fidelity Data
• Data-driven discovery of partial differential equation models with latent variables
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs
• A Latent Restoring Force Approach to Nonlinear System Identification
• Bayesian model and dimension reduction for uncertainty propagation: applications in random media
• Data-Driven Finite Elements Methods: Machine Learning Acceleration of Goal-Oriented Computations
• A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networks
• Data-driven Control of Agent-based Models: an Equation/Variable-free Machine Learning Approach
• A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables
• Constraining Gaussian processes for physics-informed acoustic emission mapping
• WeakIdent: Weak formulation for Identifying Differential Equations using Narrow-fit and Trimming
• Thermodynamics-informed neural networks for physically realistic mixed reality
• Gaussian Process Assisted Active Learning of Physical Laws
• Refinement of polygonal grids using Convolutional Neural Networks with applications to polygonal Discontinuous Galerkin and Virtual Element methods
• A Levenberg-Marquardt algorithm for sparse identification of dynamical systems
• Dynamically orthogonal tensor methods for high-dimensional nonlinear PDEs
• Physics-informed neural networks for gravity currents reconstruction from limited data
• Physics-Informed Machine Learning for Grade Prediction in Froth Flotation
• NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data
• A multiscale neural network based on hierarchical matrices
• IDENT: Identifying Differential Equations with Numerical Time evolution
• Machine Learning based refinement strategies for polyhedral grids with applications to Virtual Element and polyhedral Discontinuous Galerkin methods
• Image Inversion and Uncertainty Quantification for Constitutive Laws of Pattern Formation
• Data-driven Identification of 2D Partial Differential Equations using extracted physical features
• Data-driven, multi-moment fluid modeling of Landau damping
• Dynamic tensor approximation of high-dimensional nonlinear PDEs
• Bridging Physics-based and Data-driven modeling for Learning Dynamical Systems
• SNODE: Spectral Discretization of Neural ODEs for System Identification
• Dynamical System Identification, Model Selection and Model Uncertainty Quantification by Bayesian Inference
• Data-driven Prediction of General Hamiltonian Dynamics via Learning Exactly-Symplectic Maps
• Exponential ReLU Neural Network Approximation Rates for Point and Edge Singularities
• Learning fluid physics from highly turbulent data using sparse physics-informed discovery of empirical relations (SPIDER)
• A machine learning framework for data driven acceleration of computations of differential equations
• Symbolic Regression as Feature Engineering Method for Machine and Deep Learning Regression Tasks
• Learning and correcting non-Gaussian model errors
• Parsimony-Enhanced Sparse Bayesian Learning for Robust Discovery of Partial Differential Equations
• RAR-PINN algorithm for the data-driven vector-soliton solutions and parameter discovery of coupled nonlinear equations
• Physics-informed Information Field Theory for Modeling Physical Systems with Uncertainty Quantification
• A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning
• Deep Hamiltonian networks based on symplectic integrators
• A deep learning method for solving high-order nonlinear soliton equation
• Low-rank statistical finite elements for scalable model-data synthesis
• Nonlinear Discrete-Time Observers with Physics-Informed Neural Networks
• Stochastic stiffness identification and response estimation of Timoshenko beams via physics-informed Gaussian processes
• Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks
• Highly-scalable, physics-informed GANs for learning solutions of stochastic PDEs
• Stability selection enables robust learning of partial differential equations from limited noisy data
• Physics-integrated machine learning: embedding a neural network in the Navier-Stokes equations. Part I
• Physics-integrated machine learning: embedding a neural network in the Navier-Stokes equations. Part II
• Interface learning in fluid dynamics: statistical inference of closures within micro-macro coupling models
• System Identification Through Lipschitz Regularized Deep Neural Networks
• Learning orbital dynamics of binary black hole systems from gravitational wave measurements
• POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition
• Surface Similarity Parameter: A New Machine Learning Loss Metric for Oscillatory Spatio-Temporal Data
• Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems
• Learning Thermodynamically Stable and Galilean Invariant Partial Differential Equations for Non-equilibrium Flows
• Modelling parametric uncertainty in PDEs models via Physics-Informed Neural Networks
• Physics-informed Tensor-train ConvLSTM for Volumetric Velocity Forecasting of Loop Current
• Active Image Synthesis for Efficient Labeling
• Convergence rate of DeepONets for learning operators arising from advection-diffusion equations
• Stability analysis of hierarchical tensor methods for time-dependent PDEs
• Knowledge-integrated AutoEncoder Model
• GoRINNs: Godunov-Riemann Informed Neural Networks for Learning Hyperbolic Conservation Laws
• Explicit physics-informed neural networks for non-linear upscaling closure: the case of transport in tissues
• Physics-agnostic and Physics-infused machine learning for thin films flows: modeling, and predictions from small data
• Physics Informed Neural Networks for Modeling of 3D Flow-Thermal Problems with Sparse Domain Data
• A Machine Learning approach to enhance the SUPG stabilization method for advection-dominated differential problems
• Data-driven Discovery of Partial Differential Equations for Multiple-Physics Electromagnetic Problem
• Physics-Aware Neural Implicit Solvers for multiscale, parametric PDEs with applications in heterogeneous media
• A Priori Analysis of Stable Neural Network Solutions to Numerical PDEs
• On the Convergence of the SINDy Algorithm
• Machine learning depinning of dislocation pileups
• Probabilistic learning inference of boundary value problem with uncertainties based on Kullback-Leibler divergence under implicit constraints
• Structure Preservation for the Deep Neural Network Multigrid Solver
• Solving Optical Tomography with Deep Learning
• Physics-Informed Deep Learning for Traffic State Estimation
• Machine Discovery of Partial Differential Equations from Spatiotemporal Data
• Data assimilation empowered neural network parameterizations for subgrid processes in geophysical flows
• Sparse Symplectically Integrated Neural Networks
• Computational model discovery with reinforcement learning
• Data-driven femtosecond optical soliton excitations and parameters discovery of the high-order NLSE using the PINN
• Physics-informed Deep Learning to Solve Three-dimensional Terzaghi Consolidation Equation: Forward and Inverse Problems
• Multigrid-Augmented Deep Learning Preconditioners for the Helmholtz Equation using Compact Implicit Layers
• Machine learning reveals strain-rate-dependent predictability of discrete dislocation plasticity
• Learning of viscosity functions in rarefied gas flows with physics-informed neural networks
• Data-Driven Modeling of Nonlinear Traveling Waves
• Variational PINNs with tree-based integration and boundary element data in the modeling of multi-phase architected materials
• Learning the Tangent Space of Dynamical Instabilities from Data
• Promoting global stability in data-driven models of quadratic nonlinear dynamics
• Phase-field modeling of microstructure evolution: Recent applications, perspectives and challenges
• Machine Learning for Robust Identification of Complex Nonlinear Dynamical Systems: Applications to Earth Systems Modeling
• A Data Driven Method for Computing Quasipotentials
• Machine Learning of Partial Differential Equations from Noise Data
• Solving Elliptic Equations with Brownian Motion: Bias Reduction and Temporal Difference Learning
• Prediction of Ultrasonic Guided Wave Propagation in Solid-fluid and their Interface under Uncertainty using Machine Learning
• Data-assisted, physics-informed propagators for recurrent flows
• Bayesian Learning of Coupled Biogeochemical-Physical Models
• Linearly Constrained Gaussian Processes with Boundary Conditions
• Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint
• Particles to Partial Differential Equations Parsimoniously
• Polynomial Ridge Flowfield Estimation
• Deep neural network for solving differential equations motivated by Legendre-Galerkin approximation
• Neural Vortex Method: from Finite Lagrangian Particles to Infinite Dimensional Eulerian Dynamics
• An operator preconditioning perspective on training in physics-informed machine learning
• Computational Rational Engineering and Development: Synergies and Opportunities
• On the equivalence of molecular graph convolution and molecular wave function with poor basis set
• Robust Data-Driven Discovery of Partial Differential Equations under Uncertainties
• FD-Net with Auxiliary Time Steps: Fast Prediction of PDEs using Hessian-Free Trust-Region Methods
• Artificial neural network approach for turbulence models: A local framework
• RODE-Net: Learning Ordinary Differential Equations with Randomness from Data
• Physics-Guided Discovery of Highly Nonlinear Parametric Partial Differential Equations
• Accurately Solving Physical Systems with Graph Learning
• A composable autoencoder-based iterative algorithm for accelerating numerical simulations
• Stochastic Inference of Plate Bending from Heterogeneous Data: Physics-informed Gaussian Processes via Kirchhoff-Love Theory
• Notes on Exact Boundary Values in Residual Minimisation
• The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation
• HJB Optimal Feedback Control with Deep Differential Value Functions and Action Constraints
• Runge-Kutta Physics Informed Neural Networks: Formulation and Analysis
• Physics-informed AI and ML-based sparse system identification algorithm for discovery of PDE's representing nonlinear dynamic systems
• Physics-informed Gaussian process model for Euler-Bernoulli beam elements
• Deep Ritz-Finite Element methods: Neural Network Methods trained with Finite Elements
• Deep learning reveals hidden interactions in complex systems
• Deep neural networks for geometric multigrid methods
• On Theory-training Neural Networks to Infer the Solution of Highly Coupled Differential Equations
• Principled interpolation of Green's functions learned from data
• A Model-Constrained Tangent Slope Learning Approach for Dynamical Systems
• Physics-based Deep Learning
• Data-driven discovery of dynamics from time-resolved coherent scattering
• Physics Informed Neural Networks (PINNs)for approximating nonlinear dispersive PDEs
• Geometry encoding for numerical simulations
• Non-intrusive inference reduced order model for fluids using linear multistep neural network
• Uncertainty Quantification of Locally Nonlinear Dynamical Systems using Neural Networks
• SRoll3: A neural network approach to reduce large-scale systematic effects in the Planck High Frequency Instrument maps
• A multiscale neural network based on hierarchical nested bases
• Suppressing Modulation Instability with Reinforcement Learning
• Generalization Error Analysis of Neural networks with Gradient Based Regularization
• On the approximation of functions by tanh neural networks
• Explicit Estimation of Derivatives from Data and Differential Equations by Gaussian Process Regression
• A Neural Network Ensemble Approach to System Identification
• Methods to Recover Unknown Processes in Partial Differential Equations Using Data
• Machine learning for adjoint vector in aerodynamic shape optimization
• Sensitivity Analysis of Lift and Drag Coefficients for Flow over Elliptical Cylinders of Arbitrary Aspect Ratio and Angle of Attack using Neural Network
• Finite Difference Neural Networks: Fast Prediction of Partial Differential Equations
• Physics-Informed Machine Learning Models for Predicting the Progress of Reactive-Mixing
• Theory-training deep neural networks for an alloy solidification benchmark problem
• A Temporal Kernel Approach for Deep Learning with Continuous-time Information
• Similarity-Based Equational Inference in Physics
• A Latent space solver for PDE generalization
• Towards a Machine Learning Solution for Hubble Tension: Physics-Informed Neural Network (PINN) Analysis of Tsallis Holographic Dark Energy in Presence of Neutrinos
• Multiscale and Nonlocal Learning for PDEs using Densely Connected RNNs
• Reversible and irreversible bracket-based dynamics for deep graph neural networks
• Differentiable Spline Approximations
• Weakly-supervised learning on Schrodinger equation
• Learning the gravitational force law and other analytic functions
• Invertible Surrogate Models: Joint surrogate modelling and reconstruction of Laser-Wakefield Acceleration by invertible neural networks
• Interface Retaining Coarsening of Multiphase Flows
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs
• Object classification in analytical chemistry via data-driven discovery of partial differential equations
• Stability in Training PINNs for Stiff PDEs: Why Initial Conditions Matter
• Deep learning for gradient flows using the Brezis-Ekeland principle
• Connecting implicit and explicit large eddy simulations of two-dimensional turbulence through machine learning