20 citations · 22 across the 15 of their papers we have counts for
15 papers
How Hard is it to Rig a Benchmark? A Social Choice Analysis of Leaderboard Robustness
Polina Gordienko, Georg Schollmeyer, Frauke Kreuter +1
Multi-task benchmarks have become a central pillar of machine learning research, yet their growing influence has incentivised benchmark gaming -- strategic actions taken to improve…
Beyond Arrow: From Impossibility to Possibilities in Multi-Criteria Benchmarking
Polina Gordienko, Christoph Jansen, Julian Rodemann +1
Modern benchmarks such as HELM MMLU account for multiple metrics like accuracy, robustness and efficiency. When trying to turn these metrics into a single ranking, natural aggregat…
Union-Free Generic Depth for Non-Standard Data
Hannah Blocher, Georg Schollmeyer
Non-standard data, which fall outside classical statistical data formats, challenge state-of-the-art analysis. Examples of non-standard data include partial orders and mixed catego…
Reciprocal Learning
Julian Rodemann, Christoph Jansen, Georg Schollmeyer
We demonstrate that a wide array of machine learning algorithms are specific instances of one single paradigm: reciprocal learning. These instances range from active learning over…
Statistical Multicriteria Benchmarking via the GSD-Front
Christoph Jansen, Georg Schollmeyer, Julian Rodemann +2
Given the vast number of classifiers that have been (and continue to be) proposed, reliable methods for comparing them are becoming increasingly important. The desire for reliabili…
Data depth functions for non-standard data by use of formal concept analysis
Hannah Blocher, Georg Schollmeyer
In this article we introduce a notion of depth functions for data types that are not given in standard statistical data formats. We focus on data that cannot be represented by one…
Comparing Machine Learning Algorithms by Union-Free Generic Depth
Hannah Blocher, Georg Schollmeyer, Malte Nalenz +1
We propose a framework for descriptively analyzing sets of partial orders based on the concept of depth functions. Despite intensive studies in linear and metric spaces, there is v…
Robust Statistical Comparison of Random Variables with Locally Varying Scale of Measurement
Christoph Jansen, Georg Schollmeyer, Hannah Blocher +2
Spaces with locally varying scale of measurement, like multidimensional structures with differently scaled dimensions, are pretty common in statistics and machine learning. Neverth…
A note on the connectedness property of union-free generic sets of partial orders
Georg Schollmeyer, Hannah Blocher
This short note describes and proves a connectedness property which was introduced in Blocher et al. [2023] in the context of data depth functions for partial orders. The connected…
Depth Functions for Partial Orders with a Descriptive Analysis of Machine Learning Algorithms
Hannah Blocher, Georg Schollmeyer, Christoph Jansen +1
We propose a framework for descriptively analyzing sets of partial orders based on the concept of depth functions. Despite intensive studies of depth functions in linear and metric…
In all LikelihoodS: How to Reliably Select Pseudo-Labeled Data for Self-Training in Semi-Supervised Learning
Julian Rodemann, Christoph Jansen, Georg Schollmeyer +1
Self-training is a simple yet effective method within semi-supervised learning. The idea is to iteratively enhance training data by adding pseudo-labeled data. Its generalization p…
Multi-Target Decision Making under Conditions of Severe Uncertainty
Christoph Jansen, Georg Schollmeyer, Thomas Augustin
The quality of consequences in a decision making problem under (severe) uncertainty must often be compared among different targets (goals, objectives) simultaneously. In addition,…
Statistical Comparisons of Classifiers by Generalized Stochastic Dominance
Christoph Jansen, Malte Nalenz, Georg Schollmeyer +1
Although being a crucial question for the development of machine learning algorithms, there is still no consensus on how to compare classifiers over multiple data sets with respect…
Neural network model for imprecise regression with interval dependent variables
Krasymyr Tretiak, Georg Schollmeyer, Scott Ferson
This paper presents a computationally feasible method to compute rigorous bounds on the interval-generalisation of regression analysis to account for epistemic uncertainty in the o…
Information efficient learning of complexly structured preferences: Elicitation procedures and their application to decision making under uncertainty
Christoph Jansen, Hannah Blocher, Thomas Augustin +1
In this paper we propose efficient methods for elicitation of complexly structured preferences and utilize these in problems of decision making under (severe) uncertainty. Based on…