3 papers
cs.CL2025
Balancing Quality and Variation: Spam Filtering Distorts Data Label Distributions
Eve Fleisig, Matthias Orlikowski, Philipp Cimiano +1
For machine learning datasets to accurately represent diverse opinions in a population, they must preserve variation in data labels while filtering out spam or low-quality response…
cs.CL2025
Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions
Matthias Orlikowski, Jiaxin Pei, Paul Röttger +3
People naturally vary in their annotations for subjective questions and some of this variation is thought to be due to the person's sociodemographic characteristics. LLMs have also…
cs.CL2023
Architectural Sweet Spots for Modeling Human Label Variation by the Example of Argument Quality: It's Best to Relate Perspectives!
Philipp Heinisch, Matthias Orlikowski, Julia Romberg +1
Many annotation tasks in natural language processing are highly subjective in that there can be different valid and justified perspectives on what is a proper label for a given exa…