A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems

Abstract

Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional $\mathcal{O}(10^{\ge 2})$ stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The curse of dimensionality can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods and utilize the resulting embeddings to construct a mapping to quantities of interest via polynomial chaos expansions (PCE). We refer to the general proposed approach as manifold PCE (m-PCE), where manifold corresponds to the latent space resulting from any of the studied dimension reduction methods. To investigate the capabilities and limitations of these methods we conduct numerical tests for three physics-based systems (treated as black-boxes) having high-dimensional stochastic inputs of varying complexity modeled as both Gaussian and non-Gaussian random fields to investigate the effect of the intrinsic dimensionality of input data. We demonstrate both the advantages and limitations of the unsupervised learning methods and we conclude that a suitable m-PCE model provides a cost-effective approach compared to alternative algorithms proposed in the literature, including recently proposed expensive deep neural network-based surrogates and can be readily applied for high-dimensional UQ in stochastic PDEs.

arXiv comment45 pages, 14 figures

References in corpus (26)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• The generalization of Latin hypercube sampling
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Sparse Polynomial Chaos Expansions: Literature Survey and Benchmark
• Least angle and ${\ell }_1$ penalized regression: A review
• Solving parametric PDE problems with artificial neural networks
• Exploiting Active Subspaces to Quantify Uncertainty in the Numerical Simulation of the HyShot II Scramjet
• Data-driven polynomial chaos expansion for machine learning regression
• Functional Multi-Layer Perceptron: a Nonlinear Tool for Functional Data Analysis
• Spectral tensor-train decomposition
• A Generalized Approximate Control Variate Framework for Multifidelity Uncertainty Quantification
• Non-intrusive Low-Rank Separated Approximation of High-Dimensional Stochastic Models
• A generalized sampling and preconditioning scheme for sparse approximation of polynomial chaos expansions
• Compressive sensing adaptation for polynomial chaos expansions
• Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models
• Data-driven surrogates for high dimensional models using Gaussian process regression on the Grassmann manifold
• Reduced Wiener Chaos representation of random fields via basis adaptation and projection
• Basis Adaptive Sample Efficient Polynomial Chaos (BASE-PC)
• A gradient enhanced ${\ell }_1$-minimization for sparse approximation of polynomial chaos expansions
• Gradient-based Optimization for Regression in the Functional Tensor-Train Format
• Robust Adaptive Least Squares Polynomial Chaos Expansions in High-Frequency Applications