4 papers
On the Calibration of Crowdsourcing Aggregators
Gabriel Singer, Reda Arab, Maïssa Frikha +4
Calibration has been extensively studied for classifiers, but remains largely unexplored for crowdsourcing aggregation methods, which are increasingly used for making probabilistic…
A Model for Imbalanced Label Aggregation: A Focus on Minority-Class Detection
Gabriel Singer, Samuel Gruffaz, Olivier Vo Van +2
We study imbalanced crowdsourcing with a focus on class-dependent annotator accuracy, a setting that, to the best of our knowledge, remains relatively underexplored despite its imp…
Optimal Fair Aggregation of Crowdsourced Noisy Labels using Demographic Parity Constraints
Gabriel Singer, Samuel Gruffaz, Olivier Vo Van +2
As acquiring reliable ground-truth labels is usually costly, or infeasible, crowdsourcing and aggregation of noisy human annotations is the typical resort. Aggregating subjective l…
From Data to Rewards: a Bilevel Optimization Perspective on Maximum Likelihood Estimation
Abdelhakim Benechehab, Gabriel Singer, Corentin Léger +5
Generative models form the backbone of modern machine learning, underpinning state-of-the-art systems in text, vision, and multimodal applications. While Maximum Likelihood Estimat…