40 citations · 54 across the 9 of their papers we have counts for
9 papers
-PFN: Fast Entropy Search via In-Context Learning
Herilalaina Rakotoarison, Steven Adriaensen, Tom Viering +4
Information-theoretic acquisition functions such as Entropy Search (ES) offer a principled exploration-exploitation framework for Bayesian optimization (BO). However, their practic…
An Open-Source Training Dataset for Foundation Models for Black-box Optimization
Aaron Klein, Herilalaina Rakotoarison, Luca Thale-Bombien +1
Most black-box optimization methods require extensive hyperparameter tuning, often limiting their ability to generalize across different optimization domains. Foundation models for…
When is Warmstarting Effective for Scaling Language Models?
Neeratyoy Mallik, Maciej Janowski, Johannes Hog +4
Model growth from a given checkpoint aims to accelerate training of a larger model, offering potential resource savings. Despite recent interest, warmstarting has seen limited prac…
Warmstarting for Scaling Language Models
Neeratyoy Mallik, Maciej Janowski, Johannes Hog +4
Scaling model sizes to scale performance has worked remarkably well for the current large language models paradigm. The research and empirical findings of various scaling studies l…
In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization
Herilalaina Rakotoarison, Steven Adriaensen, Neeratyoy Mallik +3
With the increasing computational costs associated with deep learning, automated hyperparameter optimization methods, strongly relying on black-box Bayesian optimization (BO), face…
Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks
Steven Adriaensen, Herilalaina Rakotoarison, Samuel Müller +1
Learning curve extrapolation aims to predict model performance in later epochs of training, based on the performance in earlier epochs. In this work, we argue that, while the inher…
Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking
Laurent Meunier, Herilalaina Rakotoarison, Pak Kan Wong +5
Existing studies in black-box optimization for machine learning suffer from low generalizability, caused by a typically selective choice of problem instances used for training and…
Distribution-Based Invariant Deep Networks for Learning Meta-Features
Gwendoline De Bie, Herilalaina Rakotoarison, Gabriel Peyré +1
Recent advances in deep learning from probability distributions successfully achieve classification or regression from distribution samples, thus invariant under permutation of the…
Automated Machine Learning with Monte-Carlo Tree Search
Herilalaina Rakotoarison, Marc Schoenauer, Michèle Sebag
The AutoML task consists of selecting the proper algorithm in a machine learning portfolio, and its hyperparameter values, in order to deliver the best performance on the dataset a…