9 papers
JudgeArena: A Unified Framework for Reproducible LLM-Judge Evaluation
Erlis Lushtaku, Bora Kargi, Ali Elganzory +4
LLM-as-a-judge evaluation has become a dominant paradigm for ranking language models, yet the ecosystem remains fragmented: most benchmarks ship their own code base, hardcode a spe…
From Uncertain Judgments to Calibrated Rankings: Conformal Elo Estimation for LLM Evaluation
Bora Kargi, David Salinas
Evaluating new large language models typically requires costly human annotation campaigns at scale. LLM-as-a-judge offers a cheaper alternative, but judge scores carry systematic e…
ARLBench: Flexible and Efficient Benchmarking for Hyperparameter Optimization in Reinforcement Learning
Jannis Becktepe, Julian Dierkes, Carolin Benjamins +7
Hyperparameters are a critical factor in reliably training well-performing reinforcement learning (RL) agents. Unfortunately, developing and evaluating automated approaches for tun…
GAMformer: Bridging Tabular Foundation Models and Interpretable Machine Learning
Andreas Mueller, Julien Siems, Harsha Nori +4
While interpretability is crucial for machine learning applications in safety-critical domains and for regulatory compliance, existing tabular foundation models like TabPFN lack tr…
Improving LLM-based Global Optimization with Search Space Partitioning
Andrej Schwanke, Lyubomir Ivanov, David Salinas +4
Large Language Models (LLMs) have recently emerged as effective surrogate models and candidate generators within global optimization frameworks for expensive blackbox functions. De…
From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting
Shi Bin Hoo, Samuel Müller, David Salinas +1
Recent progress in foundation models has enabled strong zero-shot performance for time series forecasting. In this work, we show that such capabilities can also emerge from tabular…