Nonlinear dimension reduction for surrogate modeling using gradient information

Abstract

We introduce a method for the nonlinear dimension reduction of a high-dimensional function $u:{\mathbb{R}}^d\to \mathbb{R}$, $d\gg 1$. Our objective is to identify a nonlinear feature map $g:{\mathbb{R}}^d\to {\mathbb{R}}^m$, with a prescribed intermediate dimension $m\ll d$, so that $u$ can be well approximated by $f\circ g$ for some profile function $f:{\mathbb{R}}^m\to \mathbb{R}$. We propose to build the feature map by aligning the Jacobian $\nabla g$ with the gradient $\nabla u$, and we theoretically analyze the properties of the resulting $g$. Once $g$ is built, we construct $f$ by solving a gradient-enhanced least squares problem. Our practical algorithm makes use of a sample $\{x^{(i)},u(x^{(i)}),\nabla u(x^{(i)}){\}}_{i=1}^N$ and builds both $g$ and $f$ on adaptive downward-closed polynomial spaces, using cross validation to avoid overfitting. We numerically evaluate the performance of our algorithm across different benchmarks, and explore the impact of the intermediate dimension $m$. We show that building a nonlinear feature map $g$ can permit more accurate approximation of $u$ than a linear $g$, for the same input data set.

References in corpus (7)

• Active subspace methods in theory and practice: applications to kriging surfaces
• Kernel dimension reduction in regression
• Effective dynamics using conditional expectations
• A general theory for nonlinear sufficient dimension reduction: Formulation and estimation
• On Polynomial Chaos Expansion via Gradient-enhanced ${\ell }_1$-minimization
• Data-driven polynomial ridge approximation using variable projection
• Learning with tree-based tensor formats

Cited by in corpus (3)

• Manifold learning-based polynomial chaos expansions for high-dimensional surrogate models
• On the representation and learning of monotone triangular transport maps
• Prior normalization for certified likelihood-informed subspace detection of Bayesian inverse problems