14 citations · 14 across the 4 of their papers we have counts for
4 papers
From Global Benchmarks to Local Evaluations: Benchmarking LLMs for the German Public Sector
Camilla Dalerci, Thilo Michael, Robin Schaefer +1
Public institutions face a persistent challenge in selecting LLMs suited to their specific context. Existing benchmarks, however, are of limited use as they primarily reflect Engli…
MÖVE: A Holistic LLM Benchmark for the German Public Sector
Camilla Dalerci, Thilo Michael, Robin Schaefer +1
We present MÖVE (Modelle für die Öffentliche Verwaltung Evaluieren), a holistic benchmark for evaluating large language models (LLMs) in the context of the German public sector. Wh…
Factors in Crowdsourcing for Evaluation of Complex Dialogue Systems
Annalena Aicher, Stefan Hillmann, Isabel Feustel +3
In the last decade, crowdsourcing has become a popular method for conducting quantitative empirical studies in human-machine interaction. The remote work on a given task in crowdwo…
Bias-Aware Loss for Training Image and Speech Quality Prediction Models from Multiple Datasets
Gabriel Mittag, Saman Zadtootaghaj, Thilo Michael +2
The ground truth used for training image, video, or speech quality prediction models is based on the Mean Opinion Scores (MOS) obtained from subjective experiments. Usually, it is…