4 papers
Warmstarting for Scaling Language Models
Neeratyoy Mallik, Maciej Janowski, Johannes Hog +4
Scaling model sizes to scale performance has worked remarkably well for the current large language models paradigm. The research and empirical findings of various scaling studies l…
Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking
Laurent Meunier, Herilalaina Rakotoarison, Pak Kan Wong +5
Existing studies in black-box optimization for machine learning suffer from low generalizability, caused by a typically selective choice of problem instances used for training and…
Distribution-Based Invariant Deep Networks for Learning Meta-Features
Gwendoline De Bie, Herilalaina Rakotoarison, Gabriel Peyré +1
Recent advances in deep learning from probability distributions successfully achieve classification or regression from distribution samples, thus invariant under permutation of the…
Automated Machine Learning with Monte-Carlo Tree Search
Herilalaina Rakotoarison, Marc Schoenauer, Michèle Sebag
The AutoML task consists of selecting the proper algorithm in a machine learning portfolio, and its hyperparameter values, in order to deliver the best performance on the dataset a…