Quality and Agreement in Multilabel Emotion Annotation: A Case Study and Evaluation Framework

Emily Öhman, Anna Koufakou

Emotion annotation is inherently subjective, yet most NLP pipelines still assume "gold" labels, typically produced by majority voting, and treat annotator variation as noise. In th…