A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables

Abstract

The data-centric construction of inexpensive surrogates for fine-grained, physical models has been at the forefront of computational physics due to its significant utility in many-query tasks such as uncertainty quantification. Recent efforts have taken advantage of the enabling technologies from the field of machine learning (e.g. deep neural networks) in combination with simulation data. While such strategies have shown promise even in higher-dimensional problems, they generally require large amounts of training data even though the construction of surrogates is by definition a Small Data problem. Rather than employing data-based loss functions, it has been proposed to make use of the governing equations (in the simplest case at collocation points) in order to imbue domain knowledge in the training of the otherwise black-box-like interpolators. The present paper provides a flexible, probabilistic framework that accounts for physical structure and information both in the training objectives as well as in the surrogate model itself. We advocate a probabilistic (Bayesian) model in which equalities that are available from the physics (e.g. residuals, conservation laws) can be introduced as virtual observables and can provide additional information through the likelihood. We further advocate a generative model i.e. one that attempts to learn the joint density of inputs and outputs that is capable of making use of unlabeled data (i.e. only inputs) in a semi-supervised fashion in order to promote the discovery of lower-dimensional embeddings which are nevertheless predictive of the fine-grained model's output.

References in corpus (35)

• Adam: A Method for Stochastic Optimization
• Variational Inference: A Review for Statisticians
• The information bottleneck method
• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• Semi-Supervised Learning with Deep Generative Models
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• Machine Learning of Linear Differential Equations using Gaussian Processes
• Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• Inferring solutions of differential equations using noisy multi-fidelity data
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Randomized Iterative Methods for Linear Systems
• PDE-Net: Learning PDEs from Data
• Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks
• Variational Bayesian Inference with Stochastic Search
• A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations
• Constraint-Aware Neural Networks for Riemann Problems
• DR-RNN: A deep residual recurrent neural network for model reduction
• Physical Symmetries Embedded in Neural Networks
• Predictive Collective Variable Discovery with Deep Bayesian Models
• A physics-aware, probabilistic machine learning framework for coarse-graining high-dimensional systems in the Small Data regime
• Incorporating physical constraints in a deep probabilistic machine learning framework for coarse-graining dynamical systems
• VarNet: Variational Neural Networks for the Solution of Partial Differential Equations
• Bayesian model and dimension reduction for uncertainty propagation: applications in random media
• Query Efficient Posterior Estimation in Scientific Experiments via Bayesian Active Learning
• Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction
• Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems
• A Generalized Probabilistic Learning Approach for Multi-Fidelity Uncertainty Propagation in Complex Physical Simulations
• A Bayesian Conjugate Gradient Method
• RSN: Randomized Subspace Newton
• Physics-informed Machine Learning Method for Forecasting and Uncertainty Quantification of Partially Observed and Unobserved States in Power Grids

Cited by in corpus (7)

• Fully probabilistic deep models for forward and inverse problems in parametric PDEs
• Physics-Aware Neural Implicit Solvers for multiscale, parametric PDEs with applications in heterogeneous media
• Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling
• Variational Bayesian surrogate modelling with application to robust design optimisation
• Adversarial Disentanglement by Backpropagation with Physics-Informed Variational Autoencoder
• Manifold learning for coarse-graining atomistic simulations: Application to amorphous solids
• Reinforcement Learning Closures for Underresolved Partial Differential Equations using Synthetic Data