3 citations · 8 across the 12 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CL2021★ 3 cited
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations
Aida Mostafazadeh Davani, Mark Díaz, Vinodkumar Prabhakaran
Majority voting and averaging are common approaches employed to resolve annotator disagreements and derive single ground truth labels from multiple annotations. However, annotators…
cs.CL2021
On Releasing Annotator-Level Labels and Information in Datasets
Vinodkumar Prabhakaran, Aida Mostafazadeh Davani, Mark Díaz
A common practice in building NLP datasets, especially using crowd-sourced annotations, involves obtaining multiple annotator judgements on the same data instances, which are then…