Publications (14)
Position: EU AI Act's Research Exemptions Can Break the Publication Norms of Major AI Conferences
Alina Wernick, Kristof Meding
The EU has become one of the vanguards in regulating the digital age. A particularly important regulation in the Artificial Intelligence (AI) domain is the 2024 enacted EU AI Act.…
Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers
Thomas Klein, Sascha Meyen, Wieland Brendel +2
Benchmarking models is a key factor for the rapid progress in machine learning (ML) research. Thus, further progress depends on improving benchmarking metrics. A standard metric to…
What constitutes a Deep Fake? The blurry line between legitimate processing and manipulation under the EU AI Act
Kristof Meding, Christoph Sorge
When does a digital image resemble reality? The relevance of this question increases as the generation of synthetic images -- so called deep fakes -- becomes increasingly popular.…
Is your AI Model Accurate Enough? The Difficult Choices Behind Rigorous AI Development and the EU AI Act
Lucas G. Uberti-Bona Marin, Bram Rijsbosch, Kristof Meding +3
Technical and legal debates frequently suggest that "accuracy" is an objective, measurable, and purely technical property. We challenge this view, showing that evaluating AI perfor…
Trivial or impossible -- dichotomous data difficulty masks model differences (on ImageNet and beyond)
Kristof Meding, Luca M. Schulze Buschoff, Robert Geirhos +1
"The power of a generalization system follows directly from its biases" (Mitchell 1980). Today, CNNs are incredibly powerful generalisation systems -- but to what degree have we un…
Using predictive multiplicity to measure individual performance within the AI Act
Karolin Frohnapfel, Mara Seyfert, Sebastian Bordt +2
When building AI systems for decision support, one often encounters the phenomenon of predictive multiplicity: a single best model does not exist; instead, one can construct many m…
Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Data
Henrik Nolte, Michèle Finck, Kristof Meding
Does GPT know you? The answer depends on your level of public recognition; however, if your information was available on a website, the answer could be yes. Most Large Language Mod…
Fairness Hacking: The Malicious Practice of Shrouding Unfairness in Algorithms
Kristof Meding, Thilo Hagendorff
Fairness in machine learning (ML) is an ever-growing field of research due to the manifold potential for harm from algorithmic discrimination. To prevent such harm, a large body of…
We Need Explanation Cards to Connect Explanation Algorithms to the Real World
Eric Günther, Balázs Szabados, Kristof Meding +3
Algorithmic explanations are intended to help stakeholders understand opaque algorithmic decisions, but in practice, they often fall short. First, the meaning of algorithmic explan…
It's complicated. The relationship of algorithmic fairness and non-discrimination provisions for high-risk systems in the EU AI Act
Kristof Meding
What constitutes a fair decision? This question is not only difficult for humans but becomes more challenging when Artificial Intelligence (AI) models are used. In light of discrim…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1144
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency
Robert Geirhos, Kristof Meding, Felix A. Wichmann
A central problem in cognitive science and behavioural neuroscience as well as in machine learning and artificial intelligence research is to ascertain whether two or more decision…
Ethical Considerations and Statistical Analysis of Industry Involvement in Machine Learning Research
Thilo Hagendorff, Kristof Meding
Industry involvement in the machine learning (ML) community seems to be increasing. However, the quantitative scale and ethical implications of this influence are rather unknown. F…
Explainable AI Isn't Enough! Rethinking Algorithmic Contestability
Timo Freiesleben, Kristof Meding, Gunnar König
Machine learning systems increasingly make life-changing decisions about individuals, such as loan approvals, hiring, and cheating detection, raising a pressing question: how can i…