Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification

Abstract

State-of-the-art computer codes for simulating real physical systems are often characterized by a vast number of input parameters. Performing uncertainty quantification (UQ) tasks with Monte Carlo (MC) methods is almost always infeasible because of the need to perform hundreds of thousands or even millions of forward model evaluations in order to obtain convergent statistics. One, thus, tries to construct a cheap-to-evaluate surrogate model to replace the forward model solver. For systems with large numbers of input parameters, one has to deal with the curse of dimensionality - the exponential increase in the volume of the input space, as the number of parameters increases linearly. In this work, we demonstrate the use of deep neural networks (DNN) to construct surrogate models for numerical simulators. We parameterize the structure of the DNN in a manner that lends the DNN surrogate the interpretation of recovering a low dimensional nonlinear manifold. The model response is a parameterized nonlinear function of the low dimensional projections of the input. We think of this low dimensional manifold as a nonlinear generalization of the notion of the active subspace. Our approach is demonstrated with a problem on uncertainty propagation in a stochastic elliptic partial differential equation (SPDE) with uncertain diffusion coefficient. We deviate from traditional formulations of the SPDE problem by not imposing a specific covariance structure on the random diffusion coefficient. Instead, we attempt to solve a more challenging problem of learning a map between an arbitrary snapshot of the diffusion field and the response.

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Deep Learning in Neural Networks: An Overview
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• ADADELTA: An Adaptive Learning Rate Method
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• Weight Uncertainty in Neural Networks
• Searching for Activation Functions
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• Active subspace methods in theory and practice: applications to kriging surfaces
• An Empirical Evaluation of Deep Learning on Highway Driving
• Fractional Max-Pooling
• Practical recommendations for gradient-based training of deep architectures
• Exploiting Active Subspaces to Quantify Uncertainty in the Numerical Simulation of the HyShot II Scramjet
• Computing active subspaces with Monte Carlo
• Discovering an active subspace in a single-diode solar cell model
• Extending Expected Improvement for High-dimensional Stochastic Optimization of Expensive Black-Box Functions

Cited by in corpus (92)

• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• Deep Learning of Vortex Induced Vibrations
• Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media
• Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks
• Data Driven Governing Equations Approximation Using Deep Neural Networks
• Deep autoregressive neural networks for high-dimensional inverse problems in groundwater contaminant source identification
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• Understanding and mitigating gradient pathologies in physics-informed neural networks
• Transformers for Modeling Physical Systems
• Learning Constitutive Relations from Indirect Observations Using Deep Neural Networks
• Data-Driven Deep Learning of Partial Differential Equations in Modal Space
• Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks
• Integration of adversarial autoencoders with residual dense convolutional networks for estimation of non-Gaussian hydraulic conductivities
• A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic
• The Random Feature Model for Input-Output Maps between Banach Spaces
• Deep learning of free boundary and Stefan problems
• LaSDI: Parametric Latent Space Dynamics Identification
• 2022 Review of Data-Driven Plasma Science
• Efficient Uncertainty Quantification for Dynamic Subsurface Flow with Surrogate by Theory-guided Neural Network
• Solving inverse problems using conditional invertible neural networks
• Physics-informed neural networks for solving parametric magnetostatic problems
• Using Deep Learning to Improve Ensemble Smoother: Applications to Subsurface Characterization
• Surrogate and inverse modeling for two-phase flow in porous media via theory-guided convolutional neural network
• Generative AI and Process Systems Engineering: The Next Frontier
• On generalized residue network for deep learning of unknown dynamical systems
• Parametric Dynamic Mode Decomposition for Reduced Order Modeling
• Teaching the Incompressible Navier-Stokes Equations to Fast Neural Surrogate Models in 3D
• Learning nonlocal constitutive models with neural networks
• Structured Bayesian Gaussian process latent variable model: applications to data-driven dimensionality reduction and high-dimensional inversion
• Deep learning for thermal plasma simulation: solving 1-D arc model as an example
• A physics-aware, probabilistic machine learning framework for coarse-graining high-dimensional systems in the Small Data regime
• Manifold learning-based polynomial chaos expansions for high-dimensional surrogate models
• Coercing Machine Learning to Output Physically Accurate Results
• GINNs: Graph-Informed Neural Networks for Multiscale Physics
• A Bi-fidelity Surrogate Modeling Approach for Uncertainty Propagation in Three-Dimensional Hemodynamic Simulations
• Extended Dynamic Mode Decomposition for Inhomogeneous Problems
• Uncertainty Quantification in Three Dimensional Natural Convection using Polynomial Chaos Expansion and Deep Neural Networks
• A deep learning driven pseudospectral PCE based FFT homogenization algorithm for complex microstructures
• Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes
• Solving the Discretised Neutron Diffusion Equations using Neural Networks
• An Analysis of an Integrated Mathematical Modeling -- Artificial Neural Network Approach for the Problems with a Limited Learning Dataset
• Image Inversion and Uncertainty Quantification for Constitutive Laws of Pattern Formation
• Bi-fidelity Variational Auto-encoder for Uncertainty Quantification
• Operator Learning Using Random Features: A Tool for Scientific Computing
• Non-intrusive and semi-intrusive uncertainty quantification of a multiscale in-stent restenosis model
• Stein variational gradient descent with local approximations
• Variational Inference with NoFAS: Normalizing Flow with Adaptive Surrogate for Computationally Expensive Models
• Mutual Information for Explainable Deep Learning of Multiscale Systems
• Enforcing Deterministic Constraints on Generative Adversarial Networks for Emulating Physical Systems
• Multifidelity multilevel Monte Carlo to accelerate approximate Bayesian parameter inference for partially observed stochastic processes
• Deep Euler method: solving ODEs by approximating the local truncation error of the Euler method
• Dimensionality reduction can be used as a surrogate model for high-dimensional forward uncertainty quantification
• Using deep generative neural networks to account for model errors in Markov chain Monte Carlo inversion
• An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems
• A Bayesian neural network approach to Multi-fidelity surrogate modelling
• Performance Analysis of Semi-supervised Learning in the Small-data Regime using VAEs
• An $hp$-adaptive multi-element stochastic collocation method for surrogate modeling with information re-use
• Stochastic Modeling of Inhomogeneities in the Aortic Wall and Uncertainty Quantification using a Bayesian Encoder-Decoder Surrogate
• Power Module Heat Sink Design Optimization with Ensembles of Data-Driven Polynomial Chaos Surrogate Models
• Novel Deep neural networks for solving Bayesian statistical inverse
• Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides
• Learning Incompressible Fluid Dynamics from Scratch -- Towards Fast, Differentiable Fluid Models that Generalize
• Adaptive Sparse Polynomial Chaos Expansions via Leja Interpolation
• A Bayesian Multiscale Deep Learning Framework for Flows in Random Media
• MCMC-Net: Accelerating Markov Chain Monte Carlo with Neural Networks for Inverse Problems
• Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling
• Polynomial Chaos Expansions on Principal Geodesic Grassmannian Submanifolds for Surrogate Modeling and Uncertainty Quantification
• Improve Unscented Kalman Inversion With Low-Rank Approximation and Reduced-Order Model
• Discovering Active Subspaces for High-Dimensional Computer Models
• Leveraging Interpolation Models and Error Bounds for Verifiable Scientific Machine Learning
• A Model-Constrained Tangent Slope Learning Approach for Dynamical Systems
• Bayesian surrogate learning in dynamic simulator-based regression problems
• Surrogate-guided optimization in quantum networks
• Deep learning of parameterized equations with applications to uncertainty quantification
• Understanding ML driven HPC: Applications and Infrastructure
• Collocation approximation by deep neural ReLU networks for parametric elliptic PDEs with lognormal inputs
• Improved Surrogates in Inertial Confinement Fusion with Manifold and Cycle Consistencies
• Direct Interval Propagation Methods using Neural-Network Surrogates for Uncertainty Quantification in Physical Systems Surrogate Model
• Multilevel Surrogate-based Control Variates
• Physics-Informed Machine Learning Models for Predicting the Progress of Reactive-Mixing
• A Statistician Teaches Deep Learning
• Probabilistic partition of unity networks for high-dimensional regression problems
• Projection Pursuit Gaussian Process Regression
• A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems
• An acceleration strategy for randomize-then-optimize sampling via deep neural networks
• Level set learning with pseudo-reversible neural networks for nonlinear dimension reduction in function approximation
• Data-informed Emulators for Multi-Physics Simulations
• Efficient Characterization of Dynamic Response Variation Using Multi-Fidelity Data Fusion through Composite Neural Network
• A higher-order singular value decomposition tensor emulator for spatio-temporal simulators