Nonlinear dimension reduction for surrogate modeling using gradient information

Abstract

We introduce a method for the nonlinear dimension reduction of a high-dimensional function $u:{\mathbb{R}}^d\to \mathbb{R}$, $d\gg 1$. Our objective is to identify a nonlinear feature map $g:{\mathbb{R}}^d\to {\mathbb{R}}^m$, with a prescribed intermediate dimension $m\ll d$, so that $u$ can be well approximated by $f\circ g$ for some profile function $f:{\mathbb{R}}^m\to \mathbb{R}$. We propose to build the feature map by aligning the Jacobian $\nabla g$ with the gradient $\nabla u$, and we theoretically analyze the properties of the resulting $g$. Once $g$ is built, we construct $f$ by solving a gradient-enhanced least squares problem. Our practical algorithm makes use of a sample $\{x^{(i)},u(x^{(i)}),\nabla u(x^{(i)}){\}}_{i=1}^N$ and builds both $g$ and $f$ on adaptive downward-closed polynomial spaces, using cross validation to avoid overfitting. We numerically evaluate the performance of our algorithm across different benchmarks, and explore the impact of the intermediate dimension $m$. We show that building a nonlinear feature map $g$ can permit more accurate approximation of $u$ than a linear $g$, for the same input data set.

References in corpus (2)

• Kernel dimension reduction in regression
• A general theory for nonlinear sufficient dimension reduction: Formulation and estimation

Cited by in corpus (2)

• Manifold learning-based polynomial chaos expansions for high-dimensional surrogate models
• Prior normalization for certified likelihood-informed subspace detection of Bayesian inverse problems