Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data

Abstract

Surrogate modeling and uncertainty quantification tasks for PDE systems are most often considered as supervised learning problems where input and output data pairs are used for training. The construction of such emulators is by definition a small data problem which poses challenges to deep learning approaches that have been developed to operate in the big data regime. Even in cases where such models have been shown to have good predictive capability in high dimensions, they fail to address constraints in the data implied by the PDE model. This paper provides a methodology that incorporates the governing equations of the physical model in the loss/likelihood functions. The resulting physics-constrained, deep learning models are trained without any labeled data (e.g. employing only input data) and provide comparable predictive responses with data-driven models while obeying the constraints of the problem at hand. This work employs a convolutional encoder-decoder neural network approach as well as a conditional flow-based generative model for the solution of PDEs, surrogate model construction, and uncertainty quantification tasks. The methodology is posed as a minimization problem of the reverse Kullback-Leibler (KL) divergence between the model predictive density and the reference conditional density, where the later is defined as the Boltzmann-Gibbs distribution at a given inverse temperature with the underlying potential relating to the PDE system of interest. The generalization capability of these models to out-of-distribution input is considered. Quantification and interpretation of the predictive uncertainty is provided for a number of problems.

51 pages, 18 figures, submitted to Journal of Computational Physics

References in corpus (8)

• Conditional Generative Adversarial Nets
• On Calibration of Modern Neural Networks
• Parallel WaveNet: Fast High-Fidelity Speech Synthesis
• Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
• Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
• Diversity-Sensitive Conditional Generative Adversarial Networks
• Spherical CNNs on Unstructured Grids
• Convolutional Gaussian Processes

Cited by in corpus (42)

• Efficient training of physics-informed neural networks via importance sampling
• Understanding and mitigating gradient pathologies in physics-informed neural networks
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• Physics guided machine learning using simplified theories
• When and why PINNs fail to train: A neural tangent kernel perspective
• Physics-informed neural networks (PINNs) for fluid mechanics: A review
• Three-dimensional deep learning-based reduced order model for unsteady flow dynamics with variable Reynolds number
• On the use of graph neural networks and shape-function-based gradient computation in the deep energy method
• Neural modal ordinary differential equations: Integrating physics-based modeling with neural ordinary differential equations for modeling high-dimensional monitored structures
• Efficient nonlinear manifold reduced order model
• VarNet: Variational Neural Networks for the Solution of Partial Differential Equations
• NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation framework
• A Deep Learning Approach for the solution of Probability Density Evolution of Stochastic Systems
• Simulation free reliability analysis: A physics-informed deep learning based approach
• Physics Informed Deep Learning for Transport in Porous Media. Buckley Leverett Problem
• Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs
• A Combined Data-driven and Physics-driven Method for Steady Heat Conduction Prediction using Deep Convolutional Neural Networks
• Encoding Invariances in Deep Generative Models
• Bayesian neural networks for weak solution of PDEs with uncertainty quantification
• Physics-Constrained Bayesian Neural Network for Fluid Flow Reconstruction with Sparse and Noisy Data
• Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings
• Deep-learning-based reduced-order modeling for subsurface flow simulation
• Physics-informed Convolutional Neural Networks for Temperature Field Prediction of Heat Source Layout without Labeled Data
• Adaptive Sparse Polynomial Chaos Expansions via Leja Interpolation
• Physics Informed Deep Learning for Flow and Transport in Porous Media
• Inexact Sequential Quadratic Optimization for Minimizing a Stochastic Objective Function Subject to Deterministic Nonlinear Equality Constraints
• Learning Composable Energy Surrogates for PDE Order Reduction
• Adaptive Physics-Informed Neural Networks for Markov-Chain Monte Carlo
• Variational Autoencoding of PDE Inverse Problems
• Physics-Informed Neural Networks for Solving Multiscale Mode-Resolved Phonon Boltzmann Transport Equation
• Physics-aware, probabilistic model order reduction with guaranteed stability
• Inverse Aerodynamic Design of Gas Turbine Blades using Probabilistic Machine Learning
• Exploring Generative Physics Models with Scientific Priors in Inertial Confinement Fusion
• Application of mixed-variable physics-informed neural networks to solve normalised momentum and energy transport equations for 2D internal convective flow
• A Statistician Teaches Deep Learning
• Solving the linear transport equation by a deep neural network approach
• Reconstructing High-resolution Turbulent Flows Using Physics-Guided Neural Networks
• Data-informed Emulators for Multi-Physics Simulations
• A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems
• Uncertainty quantification for ptychography using normalizing flows
• MFPC-Net: Multi-fidelity Physics-Constrained Neural Process
• An evolve-then-correct reduced order model for hidden fluid dynamics