12 citations · 36 across the 10 of their papers we have counts for
5 papers · 1 filter
How Crowd Worker Factors Influence Subjective Annotations: A Study of Tagging Misogynistic Hate Speech in Tweets
Danula Hettiachchi, Indigo Holcombe-James, Stephanie Livingstone +4
Crowdsourced annotation is vital to both collecting labelled data to train and test automated content moderation systems and to support human-in-the-loop review of system decisions…
Designing Closed-Loop Models for Task Allocation
Vijay Keswani, L. Elisa Celis, Krishnaram Kenthapadi +1
Automatically assigning tasks to people is challenging because human performance can vary across tasks for many reasons. This challenge is further compounded in real-life settings…
Designing Closed Human-in-the-loop Deferral Pipelines
Vijay Keswani, Matthew Lease, Krishnaram Kenthapadi
In hybrid human-machine deferral frameworks, a classifier can defer uncertain cases to human decision-makers (who are often themselves fallible). Prior work on simultaneous trainin…
But Who Protects the Moderators? The Case of Crowdsourced Image Moderation
Brandon Dang, Martin J. Riedl, Matthew Lease
Though detection systems have been developed to identify obscene content such as pornography and violence, artificial intelligence is simply not good enough to fully automate this…
Design Activism for Minimum Wage Crowd Work
Akash Mankar, Riddhi J. Shah, Matthew Lease
Entry-level crowd work is often reported to pay less than minimum wage. While this may be appropriate or even necessary, due to various legal, economic, and pragmatic factors, some…