7 citations · 28 across the 9 of their papers we have counts for
11 papers
ScAn-Bench: Evaluating Scaling Analysis Methodology
Artin Sermaxhaj, Nastaran Alipour, Donat Sinani +4
Recent progress in machine learning is driven by large-scale foundation models, where scaling laws and finding optimal scaling prescriptions for architecture, data, and hyperparame…
Multi-objective Hyperparameter Optimization in the Age of Deep Learning
Soham Basu, Frank Hutter, Danny Stoll
While Deep Learning (DL) experts often have prior knowledge about which hyperparameter settings yield strong performance, only few Hyperparameter Optimization (HPO) algorithms can…
LMEMs for post-hoc analysis of HPO Benchmarking
Anton Geburek, Neeratyoy Mallik, Danny Stoll +2
The importance of tuning hyperparameters in Machine Learning (ML) and Deep Learning (DL) is established through empirical research and applications, evident from the increase in ne…
An Experimental Comparison of Partitioning Strategies for Distributed Graph Neural Network Training
Nikolai Merkel, Daniel Stoll, Ruben Mayer +1
Recently, graph neural networks (GNNs) have gained much attention as a growing area of deep learning capable of learning on graph-structured data. However, the computational and me…
PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning
Neeratyoy Mallik, Edward Bergman, Carl Hvarfner +5
Hyperparameters of Deep Learning (DL) pipelines are crucial for their downstream performance. While a large number of methods for Hyperparameter Optimization (HPO) have been develo…
Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars
Simon Schrodi, Danny Stoll, Binxin Ru +3
The discovery of neural architectures from simple building blocks is a long-standing goal of Neural Architecture Search (NAS). Hierarchical search spaces are a promising step towar…
On the Importance of Hyperparameters and Data Augmentation for Self-Supervised Learning
Diane Wagner, Fabio Ferreira, Danny Stoll +3
Self-Supervised Learning (SSL) has become a very active area of Deep Learning research where it is heavily used as a pre-training method for classification and other tasks. However…
BO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization
Carl Hvarfner, Danny Stoll, Artur Souza +3
Bayesian optimization (BO) has become an established framework and popular tool for hyperparameter optimization (HPO) of machine learning (ML) algorithms. While known for its sampl…
Winning solutions and post-challenge analyses of the ChaLearn AutoDL challenge 2019
Zhengying Liu, Adrien Pavao, Zhen Xu +22
This paper reports the results and post-challenge analyses of ChaLearn's AutoDL challenge series, which helped sorting out a profusion of AutoML solutions for Deep Learning (DL) th…
Hyperparameter Transfer Across Developer Adjustments
Danny Stoll, Jörg K. H. Franke, Diane Wagner +2
After developer adjustments to a machine learning (ML) algorithm, how can the results of an old hyperparameter optimization (HPO) automatically be used to speedup a new HPO? This q…
Learning to Design RNA
Frederic Runge, Danny Stoll, Stefan Falkner +1
Designing RNA molecules has garnered recent interest in medicine, synthetic biology, biotechnology and bioinformatics since many functional RNA molecules were shown to be involved…