Bayesian Optimization using Deep Gaussian Processes

Abstract

Bayesian Optimization using Gaussian Processes is a popular approach to deal with the optimization of expensive black-box functions. However, because of the a priori on the stationarity of the covariance matrix of classic Gaussian Processes, this method may not be adapted for non-stationary functions involved in the optimization problem. To overcome this issue, a new Bayesian Optimization approach is proposed. It is based on Deep Gaussian Processes as surrogate models instead of classic Gaussian Processes. This modeling technique increases the power of representation to capture the non-stationarity by simply considering a functional composition of stationary Gaussian Processes, providing a multiple layer structure. This paper proposes a new algorithm for Global Optimization by coupling Deep Gaussian Processes and Bayesian Optimization. The specificities of this optimization method are discussed and highlighted with academic test cases. The performance of the proposed algorithm is assessed on analytical test cases and an aerospace design optimization problem and compared to the state-of-the-art stationary and non-stationary Bayesian Optimization approaches.

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• A Tutorial on Bayesian Optimization
• GPflow: A Gaussian process library using TensorFlow
• Deep Gaussian Processes
• Deep Gaussian Processes for Regression using Approximate Expectation Propagation
• Portfolio Allocation for Bayesian Optimization
• Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo
• Variational Auto-encoded Deep Gaussian Processes
• An Entropy Search Portfolio for Bayesian Optimization
• Adaptive Rate of Convergence of Thompson Sampling for Gaussian Process Optimization

Cited by in corpus (4)

• The Issues of Mismodelling Gravitational-Wave Data for Parameter Estimation
• Heuristic-free Optimization of Force-Controlled Robot Search Strategies in Stochastic Environments
• Neural Process for Black-Box Model Optimization Under Bayesian Framework
• Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties