47 citations · 87 across the 21 of their papers we have counts for
21 papers · 1 filter
Efficiently Estimating Optimal Hyperparameter Scaling Laws through Power-Law Entropy Search
Zhiliang Chen, Sebastian Ament, David Eriksson +4
Optimal hyperparameter scaling laws describe how the best hyperparameters for large language model (LLM) training change with model and data scale, enabling practitioners to predic…
Pitfalls and Remedies for Multi-Task Bayesian Optimization
Carl Hvarfner, Sam Daulton, Max Balandat +1
Bayesian optimization routinely warm-starts a target experiment with data from related source tasks, and the multi-task Gaussian process is the textbook surrogate for the job. We r…
MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment
Hanxian Huang, Igor Fedorov, Andrey Gromov +14
Real-time AI experiences call for on-device large language models (OD-LLMs) optimized for efficient deployment on resource-constrained hardware. The most useful OD-LLMs produce nea…
Empirical Gaussian Processes
Jihao Andreas Lin, Sebastian Ament, Louis C. Tiao +3
Gaussian processes (GPs) are powerful and widely used probabilistic regression models, but their effectiveness in practice is often limited by the choice of kernel function. This k…
BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability
Samuel Daulton, David Eriksson, Maximilian Balandat +1
Bayesian optimization (BO) is a popular technique for sample-efficient optimization of black-box functions. In many applications, the parameters being tuned come with a carefully e…
Informed Initialization for Bayesian Optimization and Active Learning
Carl Hvarfner, David Eriksson, Eytan Bakshy +1
Bayesian Optimization is a widely used method for optimizing expensive black-box functions, relying on probabilistic surrogate models such as Gaussian Processes. The quality of the…