Gradient and Uncertainty Enhanced Sequential Sampling for Global Fit

Abstract

Surrogate models based on machine learning methods have become an important part of modern engineering to replace costly computer simulations. The data used for creating a surrogate model are essential for the model accuracy and often restricted due to cost and time constraints. Adaptive sampling strategies have been shown to reduce the number of samples needed to create an accurate model. This paper proposes a new sampling strategy for global fit called Gradient and Uncertainty Enhanced Sequential Sampling (GUESS). The acquisition function uses two terms: the predictive posterior uncertainty of the surrogate model for exploration of unseen regions and a weighted approximation of the second and higher-order Taylor expansion values for exploitation. Although various sampling strategies have been proposed so far, the selection of a suitable method is not trivial. Therefore, we compared our proposed strategy to 9 adaptive sampling strategies for global surrogate modeling, based on 26 different 1 to 8-dimensional deterministic benchmarks functions. Results show that GUESS achieved on average the highest sample efficiency compared to other surrogate-based strategies on the tested examples. An ablation study considering the behavior of GUESS in higher dimensions and the importance of surrogate choice is also presented.

References in corpus (13)

• Scikit-learn: Machine Learning in Python
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• A Tutorial on Principal Component Analysis
• Fourier Neural Operator for Parametric Partial Differential Equations
• Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
• A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
• Deep Gaussian Processes
• SelectNet: Self-paced Learning for High-dimensional Partial Differential Equations
• Local Latin Hypercube Refinement for Multi-objective Design Uncertainty Optimization
• Bayesian Optimization in a Billion Dimensions via Random Embeddings
• Deep Gaussian Covariance Network