7 citations · 10 across the 6 of their papers we have counts for
6 papers · 1 filter
When is Warmstarting Effective for Scaling Language Models?
Neeratyoy Mallik, Maciej Janowski, Johannes Hog +4
Model growth from a given checkpoint aims to accelerate training of a larger model, offering potential resource savings. Despite recent interest, warmstarting has seen limited prac…
How Usable is Automated Feature Engineering for Tabular Data?
Bastian Schäfer, Lennart Purucker, Maciej Janowski +1
Tabular data, consisting of rows and columns, is omnipresent across various machine learning applications. Each column represents a feature, and features can be combined or transfo…
Warmstarting for Scaling Language Models
Neeratyoy Mallik, Maciej Janowski, Johannes Hog +4
Scaling model sizes to scale performance has worked remarkably well for the current large language models paradigm. The research and empirical findings of various scaling studies l…
Regularized Neural Ensemblers
Sebastian Pineda Arango, Maciej Janowski, Lennart Purucker +3
Ensemble methods are known for enhancing the accuracy and robustness of machine learning models by combining multiple base learners. However, standard approaches like greedy or ran…
PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning
Neeratyoy Mallik, Edward Bergman, Carl Hvarfner +5
Hyperparameters of Deep Learning (DL) pipelines are crucial for their downstream performance. While a large number of methods for Hyperparameter Optimization (HPO) have been develo…
Scaling Laws for Hyperparameter Optimization
Arlind Kadra, Maciej Janowski, Martin Wistuba +1
Hyperparameter optimization is an important subfield of machine learning that focuses on tuning the hyperparameters of a chosen algorithm to achieve peak performance. Recently, the…