5 papers
CDS4RAG: Cyclic Dual-Sequential Hyperparameter Optimization for RAG
Pengzhou Chen, Tao Chen
Retrieval-Augmented Generation (RAG) is sensitive to the vast hyperparameters of the retriever and generator, yet optimizing them using given queries is a challenging task due to t…
LLMSYS-HPOBench: Hyperparameter Optimization Benchmark Suite for Real-World LLM Systems
Siyu Wu, Yulong Ye, Zezhen Xiang +3
Large Language Model (LLM) systems have been the frontier of AI in many application domains, leading to new challenges and opportunities for hyperparameter optimization (HPO) for t…
Evaluating Useful Surrogate Models for Configuration Tuning Beyond Accuracy: A Fitness Landscape Analysis Perspective
Pengzhou Chen, Hongyuan Liang, Tao Chen
To efficiently tune configuration for better software system performance (e.g., latency) at the deployment and maintenance stage, many tuners have leveraged a surrogate model to ex…
PromiseTune: Unveiling Causally Promising and Explainable Configuration Tuning
Pengzhou Chen, Tao Chen
The high configurability of modern software systems has made configuration tuning a crucial step for assuring system performance, e.g., latency or throughput. However, given the ex…
Accuracy Can Lie: On the Impact of Surrogate Model in Configuration Tuning
Pengzhou Chen, Jingzhi Gong, Tao Chen
To ease the expensive measurements during configuration tuning, it is natural to build a surrogate model as the replacement of the system, and thereby the configuration performance…