5 papers
From Ground Truth to Measurement: A Statistical Framework for Human Labeling
Robert Chew, Stephanie Eckman, Christoph Kern +1
Supervised machine learning assumes that labeled data provide accurate measurements of the concepts models are meant to learn. Yet in practice, human labeling introduces systematic…
Dialect and Gender Bias in YouTube's Spanish Captioning System
Iris Dania Jimenez, Christoph Kern
Spanish is the official language of twenty-one countries and is spoken by over 441 million people. Naturally, there are many variations in how Spanish is spoken across these countr…
Bias Begins with Data: The FairGround Corpus for Robust and Reproducible Research on Algorithmic Fairness
Jan Simson, Alessandro Fabris, Cosima Fröhner +2
As machine learning (ML) systems are increasingly adopted in high-stakes decision-making domains, ensuring fairness in their outputs has become a central challenge. At the core of…
Bias in the Loop: How Humans Evaluate AI-Generated Suggestions
Jacob Beck, Stephanie Eckman, Christoph Kern +1
Human-AI collaboration increasingly drives decision-making across industries, from medical diagnosis to content moderation. While AI systems promise efficiency gains by providing a…
Aligning NLP Models with Target Population Perspectives using PAIR: Population-Aligned Instance Replication
Stephanie Eckman, Bolei Ma, Christoph Kern +3
Models trained on crowdsourced annotations may not reflect population views, if those who work as annotators do not represent the broader population. In this paper, we propose PAIR…