4 papers
Quantifying Ambiguity in Categorical Annotations: A Measure and Statistical Inference Framework
Christopher Klugmann, Daniel Kondermann
Human-generated categorical annotations frequently produce empirical response distributions (soft labels) that reflect ambiguity rather than simple annotator error. We introduce an…
Decomposing Visual Classification: Assessing Tree-Based Reasoning in VLMs
Sary Elmansoury, Islam Mesabah, Gerrit Großmann +4
Vision language models (VLMs) excel at zero-shot visual classification, but their performance on fine-grained tasks and large hierarchical label spaces is understudied. This paper…
Minority Reports: Balancing Cost and Quality in Ground Truth Data Annotation
Hsuan Wei Liao, Christopher Klugmann, Daniel Kondermann +1
High-quality data annotation is an essential but laborious and costly aspect of developing machine learning-based software. We explore the inherent tradeoff between annotation accu…
No Need to Sacrifice Data Quality for Quantity: Crowd-Informed Machine Annotation for Cost-Effective Understanding of Visual Data
Christopher Klugmann, Rafid Mahmood, Guruprasad Hegde +2
Labeling visual data is expensive and time-consuming. Crowdsourcing systems promise to enable highly parallelizable annotations through the participation of monetarily or otherwise…