4 papers
Fine-Grained Perspectives: Modeling Explanations with Annotator-Specific Rationales
Olufunke O. Sarumi, Charles Welch, Daniel Braun
Beyond exploring disaggregated labels for modeling perspectives, annotator rationales provide fine-grained signals of individual perspectives. In this work, we propose a framework…
The Impact of Annotator Personas on LLM Behavior Across the Perspectivism Spectrum
Olufunke O. Sarumi, Charles Welch, Daniel Braun +1
In this work, we explore the capability of Large Language Models (LLMs) to annotate hate speech and abusiveness while considering predefined annotator personas within the strong-to…
Funzac at CoMeDi Shared Task: Modeling Annotator Disagreement from Word-In-Context Perspectives
Olufunke O. Sarumi, Charles Welch, Lucie Flek +1
In this work, we evaluate annotator disagreement in Word-in-Context (WiC) tasks exploring the relationship between contextual meaning and disagreement as part of the CoMeDi shared…
Corpus Considerations for Annotator Modeling and Scaling
Olufunke O. Sarumi, Béla Neuendorf, Joan Plepi +3
Recent trends in natural language processing research and annotation tasks affirm a paradigm shift from the traditional reliance on a single ground truth to a focus on individual p…