47 citations · 47 across the 10 of their papers we have counts for
10 papers
Efficiently Estimating Optimal Hyperparameter Scaling Laws through Power-Law Entropy Search
Zhiliang Chen, Sebastian Ament, David Eriksson +4
Optimal hyperparameter scaling laws describe how the best hyperparameters for large language model (LLM) training change with model and data scale, enabling practitioners to predic…
MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment
Hanxian Huang, Igor Fedorov, Andrey Gromov +14
Real-time AI experiences call for on-device large language models (OD-LLMs) optimized for efficient deployment on resource-constrained hardware. The most useful OD-LLMs produce nea…
Empirical Gaussian Processes
Jihao Andreas Lin, Sebastian Ament, Louis C. Tiao +3
Gaussian processes (GPs) are powerful and widely used probabilistic regression models, but their effectiveness in practice is often limited by the choice of kernel function. This k…
BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability
Samuel Daulton, David Eriksson, Maximilian Balandat +1
Bayesian optimization (BO) is a popular technique for sample-efficient optimization of black-box functions. In many applications, the parameters being tuned come with a carefully e…
Informed Initialization for Bayesian Optimization and Active Learning
Carl Hvarfner, David Eriksson, Eytan Bakshy +1
Bayesian Optimization is a widely used method for optimizing expensive black-box functions, relying on probabilistic surrogate models such as Gaussian Processes. The quality of the…
Scalable Gaussian Processes with Latent Kronecker Structure
Jihao Andreas Lin, Sebastian Ament, Maximilian Balandat +3
Applying Gaussian processes (GPs) to very large datasets remains a challenge due to limited computational scalability. Matrix structures, such as the Kronecker product, can acceler…
Robust Gaussian Processes via Relevance Pursuit
Sebastian Ament, Elizabeth Santorella, David Eriksson +3
Gaussian processes (GPs) are non-parametric probabilistic regression models that are popular due to their flexibility, data efficiency, and well-calibrated uncertainty estimates. H…
Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces
Aryan Deshwal, Sait Cakmak, Yuhou Xia +1
Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box functions. However, in settings with very few function evaluations, a successful app…
Approximation-Aware Bayesian Optimization
Natalie Maus, Kyurae Kim, Geoff Pleiss +3
High-dimensional Bayesian optimization (BO) tasks such as molecular design often require 10,000 function evaluations before obtaining meaningful results. While methods like sparse…
Unexpected Improvements to Expected Improvement for Bayesian Optimization
Sebastian Ament, Samuel Daulton, David Eriksson +2
Expected Improvement (EI) is arguably the most popular acquisition function in Bayesian optimization and has found countless successful applications, but its performance is often e…