52 citations · 81 across the 9 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.CL2023
Annotation Imputation to Individualize Predictions: Initial Studies on Distribution Dynamics and Model Predictions
London Lowmanstone, Ruyuan Wan, Risako Owan +2
Annotating data via crowdsourcing is time-consuming and expensive. Due to these costs, dataset creators often have each annotator label only a small subset of the data. This leads…
cs.CL2023★ 2 cited
Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information
Ruyuan Wan, Jaehyung Kim, Dongyeop Kang
In NLP annotation, it is common to have multiple annotators label the text and then obtain the ground truth labels based on the agreement of major annotators. However, annotators a…