Rethinking Bayesian Optimization for Co-Optimizing LLM Training Configurations

Zhiliang Chen, Alfred Wei Lun Leong, Shao Yong Ong +6

Fine-tuning an LLM to maximize performance on a downstream task requires finding the optimal data and model configuration. This is a black-box optimization problem that Bayesian op…