12 papers
GFlowPO: Generative Flow Network as a Language Model Prompt Optimizer
Junmo Cho, Suhan Kim, Sangjune An +5
Finding effective prompts for language models (LMs) is critical yet notoriously difficult: the prompt space is combinatorially large, rewards are sparse due to expensive target-LM…
Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuning
Dong Bok Lee, Aoxuan Silvia Zhang, Byungjoo Kim +5
In this paper, we address the problem of \emph{cost-sensitive} hyperparameter optimization (HPO) built upon freeze-thaw Bayesian optimization (BO). Specifically, we assume a scenar…
Bayesian Neural Scaling Law Extrapolation with Prior-Data Fitted Networks
Dongwoo Lee, Dong Bok Lee, Steven Adriaensen +5
Scaling has been a major driver of recent advancements in deep learning. Numerous empirical studies have found that scaling laws often follow the power-law and proposed several var…
PCoreSet: Effective Active Learning through Knowledge Distillation from Vision-Language Models
Seongjae Kang, Dong Bok Lee, Hyungjoon Jang +2
Knowledge distillation (KD) is a widely used framework for training compact, task-specific models by transferring the knowledge from teacher models. However, its application to act…
Distilling LLM Agent into Small Models with Retrieval and Code Tools
Minki Kang, Jongwon Jeong, Seanie Lee +2
Large language models (LLMs) excel at complex reasoning tasks but remain computationally expensive, limiting their practical deployment. To address this, recent works have focused…
FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA
Seanie Lee, Sangwoo Park, Dong Bok Lee +5
Low-Rank Adaptation (LoRA), which introduces a product of two trainable low-rank matrices into frozen pre-trained weights, is widely used for efficient fine-tuning of language mode…