3 papers
cs.LG2025
Quantifying Ambiguity in Categorical Annotations: A Measure and Statistical Inference Framework
Christopher Klugmann, Daniel Kondermann
Human-generated categorical annotations frequently produce empirical response distributions (soft labels) that reflect ambiguity rather than simple annotator error. We introduce an…
cs.LG2025
Minority Reports: Balancing Cost and Quality in Ground Truth Data Annotation
Hsuan Wei Liao, Christopher Klugmann, Daniel Kondermann +1
High-quality data annotation is an essential but laborious and costly aspect of developing machine learning-based software. We explore the inherent tradeoff between annotation accu…
cs.HC2024
No Need to Sacrifice Data Quality for Quantity: Crowd-Informed Machine Annotation for Cost-Effective Understanding of Visual Data
Christopher Klugmann, Rafid Mahmood, Guruprasad Hegde +2
Labeling visual data is expensive and time-consuming. Crowdsourcing systems promise to enable highly parallelizable annotations through the participation of monetarily or otherwise…