7 papers
JudgeArena: A Unified Framework for Reproducible LLM-Judge Evaluation
Erlis Lushtaku, Bora Kargi, Ali Elganzory +4
LLM-as-a-judge evaluation has become a dominant paradigm for ranking language models, yet the ecosystem remains fragmented: most benchmarks ship their own code base, hardcode a spe…
Can LLMs Beat Classical Hyperparameter Optimization Algorithms? A Study on autoresearch
Fabio Ferreira, Lucca Wobbe, Arjun Krishnakumar +2
The autoresearch repository enables an LLM agent to optimize hyperparameters by editing training code directly. We use it as a testbed to compare classical HPO algorithms against L…
Improving LLM-based Global Optimization with Search Space Partitioning
Andrej Schwanke, Lyubomir Ivanov, David Salinas +4
Large Language Models (LLMs) have recently emerged as effective surrogate models and candidate generators within global optimization frameworks for expensive blackbox functions. De…
Meta-Learning and Synthetic Data for Automated Pretraining and Finetuning
Fabio Ferreira
The growing number of pretrained models in Machine Learning (ML) presents significant challenges for practitioners. Given a new dataset, they need to determine the most suitable de…
Beyond Random Augmentations: Pretraining with Hard Views
Fabio Ferreira, Ivo Rapant, Jörg K. H. Franke +1
Self-Supervised Learning (SSL) methods typically rely on random image augmentations, or views, to make models invariant to different transformations. We hypothesize that the effica…
Transfer Learning for Finetuning Large Language Models
Tobias Strangmann, Lennart Purucker, Jörg K. H. Franke +3
As the landscape of large language models expands, efficiently finetuning for specific tasks becomes increasingly crucial. At the same time, the landscape of parameter-efficient fi…