Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization
arXiv:2206.01409 · doi:10.1080/10618600.2024.2308216
Abstract
This paper presents a new type of hybrid model for Bayesian optimization (BO) adept at managing mixed variables, encompassing both quantitative (continuous and integer) and qualitative (categorical) types. Our proposed new hybrid models (named hybridM) merge the Monte Carlo Tree Search structure (MCTS) for categorical variables with Gaussian Processes (GP) for continuous ones. hybridM leverages the upper confidence bound tree search (UCTS) for MCTS strategy, showcasing the tree architecture's integration into Bayesian optimization. Our innovations, including dynamic online kernel selection in the surrogate modeling phase and a unique UCTS search strategy, position our hybrid models as an advancement in mixed-variable surrogate models. Numerical experiments underscore the superiority of hybrid models, highlighting their potential in Bayesian optimization.
33 pages, 8 Figures
References in corpus (9)
- Practical Bayesian Optimization of Machine Learning Algorithms
- 3D-HST WFC3-selected Photometric Catalogs in the Five CANDELS/3D-HST Fields: Photometry, Photometric Redshifts and Stellar Masses
- Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces
- Bayesian Inference in Monte-Carlo Tree Search
- Bayesian Optimization over Hybrid Spaces
- Non-smooth Bayesian Optimization in Tuning Problems
- Efficient and Robust Bayesian Selection of Hyperparameters in Dimension Reduction for Visualization
- Structural Kernel Search via Bayesian Optimization and Symbolical Optimal Transport
- Mixed Variable Bayesian Optimization with Frequency Modulated Kernels