5 papers
BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks
Ruth Wan Theng Chew, Zhiliang Chen, Apivich Hemachandra +1
Optimization of LLM training and inference configurations, such as hyperparameters, data mixtures, and prompts, is critical to performance, but it is often approached heuristically…
DUET: Optimizing Training Data Mixtures via Feedback from Unseen Evaluation Tasks
Zhiliang Chen, Gregory Kang Ruey Lau, Chuan-Sheng Foo +1
The performance of an LLM depends heavily on the relevance of its training data to the downstream evaluation task. However, in practice, the data involved in an unseen evaluation t…
The Chicken and Egg Dilemma: Co-optimizing Data and Model Configurations for LLMs
Zhiliang Chen, Alfred Wei Lun Leong, Shao Yong Ong +6
Co-optimizing data and model configurations for training LLMs presents a classic chicken-and-egg dilemma: The best training data configuration (e.g., data mixture) for a downstream…
Uncovering Scaling Laws for Large Language Models via Inverse Problems
Arun Verma, Zhaoxuan Wu, Zijian Zhou +15
Large Language Models (LLMs) are large-scale pretrained models that have achieved remarkable success across diverse domains. These successes have been driven by unprecedented compl…
Broaden your SCOPE! Efficient Multi-turn Conversation Planning for LLMs with Semantic Space
Zhiliang Chen, Xinyuan Niu, Chuan-Sheng Foo +1
Large language models (LLMs) are used in chatbots or AI assistants to hold conversations with a human user. In such applications, the quality (e.g., user engagement, safety) of a c…