20 citations · 22 across the 15 of their papers we have counts for
5 papers · 1 filter
How Hard is it to Rig a Benchmark? A Social Choice Analysis of Leaderboard Robustness
Polina Gordienko, Georg Schollmeyer, Frauke Kreuter +1
Multi-task benchmarks have become a central pillar of machine learning research, yet their growing influence has incentivised benchmark gaming -- strategic actions taken to improve…
Beyond Arrow: From Impossibility to Possibilities in Multi-Criteria Benchmarking
Polina Gordienko, Christoph Jansen, Julian Rodemann +1
Modern benchmarks such as HELM MMLU account for multiple metrics like accuracy, robustness and efficiency. When trying to turn these metrics into a single ranking, natural aggregat…
Comparing Machine Learning Algorithms by Union-Free Generic Depth
Hannah Blocher, Georg Schollmeyer, Malte Nalenz +1
We propose a framework for descriptively analyzing sets of partial orders based on the concept of depth functions. Despite intensive studies in linear and metric spaces, there is v…
A note on the connectedness property of union-free generic sets of partial orders
Georg Schollmeyer, Hannah Blocher
This short note describes and proves a connectedness property which was introduced in Blocher et al. [2023] in the context of data depth functions for partial orders. The connected…
Depth Functions for Partial Orders with a Descriptive Analysis of Machine Learning Algorithms
Hannah Blocher, Georg Schollmeyer, Christoph Jansen +1
We propose a framework for descriptively analyzing sets of partial orders based on the concept of depth functions. Despite intensive studies of depth functions in linear and metric…