88 citations · 101 across the 8 of their papers we have counts for
10 papers
-PFN: Fast Entropy Search via In-Context Learning
Herilalaina Rakotoarison, Steven Adriaensen, Tom Viering +4
Information-theoretic acquisition functions such as Entropy Search (ES) offer a principled exploration-exploitation framework for Bayesian optimization (BO). However, their practic…
Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuning
Dong Bok Lee, Aoxuan Silvia Zhang, Byungjoo Kim +5
In this paper, we address the problem of \emph{cost-sensitive} hyperparameter optimization (HPO) built upon freeze-thaw Bayesian optimization (BO). Specifically, we assume a scenar…
Tune My Adam, Please!
Theodoros Athanasiadis, Steven Adriaensen, Samuel Müller +1
The Adam optimizer remains one of the most widely used optimizers in deep learning, and effectively tuning its hyperparameters is key to optimizing performance. However, tuning can…
Bayesian Neural Scaling Law Extrapolation with Prior-Data Fitted Networks
Dongwoo Lee, Dong Bok Lee, Steven Adriaensen +5
Scaling has been a major driver of recent advancements in deep learning. Numerous empirical studies have found that scaling laws often follow the power-law and proposed several var…
Gompertz Linear Units: Leveraging Asymmetry for Enhanced Learning Dynamics
Indrashis Das, Mahmoud Safari, Steven Adriaensen +1
Activation functions are fundamental elements of deep learning architectures as they significantly influence training dynamics. ReLU, while widely used, is prone to the dying neuro…
In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization
Herilalaina Rakotoarison, Steven Adriaensen, Neeratyoy Mallik +3
With the increasing computational costs associated with deep learning, automated hyperparameter optimization methods, strongly relying on black-box Bayesian optimization (BO), face…
Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks
Steven Adriaensen, Herilalaina Rakotoarison, Samuel Müller +1
Learning curve extrapolation aims to predict model performance in later epochs of training, based on the performance in earlier epochs. In this work, we argue that, while the inher…
Automated Dynamic Algorithm Configuration
Steven Adriaensen, André Biedenkapp, Gresa Shala +4
The performance of an algorithm often critically depends on its parameter configuration. While a variety of automated algorithm configuration methods have been proposed to relieve…
DACBench: A Benchmark Library for Dynamic Algorithm Configuration
Theresa Eimer, André Biedenkapp, Maximilian Reimer +3
Dynamic Algorithm Configuration (DAC) aims to dynamically control a target algorithm's hyperparameters in order to improve its performance. Several theoretical and empirical result…
Metaheuristics "In the Large"
Jerry Swan, Steven Adriaensen, Alexander E. I. Brownlee +13
Following decades of sustained improvement, metaheuristics are one of the great success stories of optimization research. However, in order for research in metaheuristics to avoid…