Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks

Abstract

In recent years, deep learning has proven to be a viable methodology for surrogate modeling and uncertainty quantification for a vast number of physical systems. However, in their traditional form, such models can require a large amount of training data. This is of particular importance for various engineering and scientific applications where data may be extremely expensive to obtain. To overcome this shortcoming, physics-constrained deep learning provides a promising methodology as it only utilizes the governing equations. In this work, we propose a novel auto-regressive dense encoder-decoder convolutional neural network to solve and model non-linear dynamical systems without training data at a computational cost that is potentially magnitudes lower than standard numerical solvers. This model includes a Bayesian framework that allows for uncertainty quantification of the predicted quantities of interest at each time-step. We rigorously test this model on several non-linear transient partial differential equation systems including the turbulence of the Kuramoto-Sivashinsky equation, multi-shock formation and interaction with 1D Burgers' equation and 2D wave dynamics with coupled Burgers' equations. For each system, the predictive results and uncertainty are presented and discussed together with comparisons to the results obtained from traditional numerical analysis methods.

arXiv comment48 pages, 30 figures, Accepted to Journal of Computational Physics

References in corpus (29)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Decoupled Weight Decay Regularization
• Deep Residual Learning for Image Recognition
• On the difficulty of training Recurrent Neural Networks
• Geometric deep learning: going beyond Euclidean data
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• Weight Uncertainty in Neural Networks
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• Density estimation using Real NVP
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
• A Simple Baseline for Bayesian Uncertainty in Deep Learning
• Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks
• A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
• Quantifying model form uncertainty in Reynolds-averaged turbulence models with Bayesian deep neural networks
• Estimating Uncertainties in Statistics Computed from DNS
• Long-time predictive modeling of nonlinear dynamical systems using neural networks
• A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations
• On the state space geometry of the Kuramoto-Sivashinsky flow in a periodic domain
• Predictive Collective Variable Discovery with Deep Bayesian Models

Cited by in corpus (56)

• PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain
• CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method
• Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
• Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems
• PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs
• Understanding and mitigating gradient pathologies in physics-informed neural networks
• Super-resolution and denoising of fluid flow using physics-informed convolutional neural networks without high-resolution labels
• Transformers for Modeling Physical Systems
• Deep Learning in Deterministic Computational Mechanics
• KAN-ODEs: Kolmogorov-Arnold Network Ordinary Differential Equations for Learning Dynamical Systems and Hidden Physics
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• Physics guided machine learning using simplified theories
• Wavefield reconstruction inversion via physics-informed neural networks
• Multiphase flow prediction with deep neural networks
• Characterizing possible failure modes in physics-informed neural networks
• What Are Bayesian Neural Network Posteriors Really Like?
• Solving inverse problems using conditional invertible neural networks
• Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics
• Reservoir computing model of two-dimensional turbulent convection
• Model fusion with physics-guided machine learning
• Teaching the Incompressible Navier-Stokes Equations to Fast Neural Surrogate Models in 3D
• Mosaic Flows: A Transferable Deep Learning Framework for Solving PDEs on Unseen Domains
• Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problems
• A neural network multigrid solver for the Navier-Stokes equations
• On the use of graph neural networks and shape-function-based gradient computation in the deep energy method
• Accelerating hydrodynamic simulations of urban drainage systems with physics-guided machine learning
• Attention U-Net as a surrogate model for groundwater prediction
• An unsupervised latent/output physics-informed convolutional-LSTM network for solving partial differential equations using peridynamic differential operator
• A finite element-based physics-informed operator learning framework for spatiotemporal partial differential equations on arbitrary domains
• On the Use of Neural Networks for Full Waveform Inversion
• Data-Driven Discovery of Coarse-Grained Equations
• Can Transfer Neuroevolution Tractably Solve Your Differential Equations?
• Extended Dynamic Mode Decomposition for Inhomogeneous Problems
• CEENs: Causality-enforced evolutional networks for solving time-dependent partial differential equations
• PCNN: A physics-constrained neural network for multiphase flows
• A Combined Data-driven and Physics-driven Method for Steady Heat Conduction Prediction using Deep Convolutional Neural Networks
• Bi-fidelity Variational Auto-encoder for Uncertainty Quantification
• Bayesian neural networks for weak solution of PDEs with uncertainty quantification
• Physics-Constrained Bayesian Neural Network for Fluid Flow Reconstruction with Sparse and Noisy Data
• Improved Surrogate Modeling of Fluid Dynamics with Physics-Informed Neural Networks
• Hard Encoding of Physics for Learning Spatiotemporal Dynamics
• LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data
• Learning continuous-time PDEs from sparse data with graph neural networks
• Learning Differential Operators for Interpretable Time Series Modeling
• Physics Encoded Blocks in Residual Neural Network Architectures for Digital Twin Models
• Physics-Informed Graph-Mesh Networks for PDEs: A hybrid approach for complex problems
• Learning Incompressible Fluid Dynamics from Scratch -- Towards Fast, Differentiable Fluid Models that Generalize
• Uncertainty Quantification of Surrogate Models using Conformal Prediction
• A Bayesian Multiscale Deep Learning Framework for Flows in Random Media
• A Deep Learning Approach for Predicting Spatiotemporal Dynamics From Sparsely Observed Data
• Physics-Guided Discovery of Highly Nonlinear Parametric Partial Differential Equations
• Bayesian Reasoning for Physics Informed Neural Networks
• Physics-Informed Time-Integrated DeepONet: Temporal Tangent Space Operator Learning for High-Accuracy Inference
• Variational inference formulation for a model-free simulation of a dynamical system with unknown parameters by a recurrent neural network
• Physics-Informed Machine Learning Models for Predicting the Progress of Reactive-Mixing
• Orthogonal Transforms in Neural Networks Amount to Effective Regularization