Breaking Monotony with Meaning: Motivation in Crowdsourcing Markets

Abstract

We conduct the first natural field experiment to explore the relationship between the "meaningfulness" of a task and worker effort. We employed about 2,500 workers from Amazon's Mechanical Turk (MTurk), an online labor market, to label medical images. Although given an identical task, we experimentally manipulated how the task was framed. Subjects in the meaningful treatment were told that they were labeling tumor cells in order to assist medical researchers, subjects in the zero-context condition (the control group) were not told the purpose of the task, and, in stark contrast, subjects in the shredded treatment were not given context and were additionally told that their work would be discarded. We found that when a task was framed more meaningfully, workers were more likely to participate. We also found that the meaningful treatment increased the quantity of output (with an insignificant change in quality) while the shredded treatment decreased the quality of output (with no change in quantity). We believe these results will generalize to other short-term labor markets. Our study also discusses MTurk as an exciting platform for running natural field experiments in economics.

26 pages, 15 figures, 4 tables

Cited by in corpus (11)

• Statistical Inference for Covariate-Adaptive Randomization Procedures
• A Glimpse Far into the Future: Understanding Long-term Crowd Worker Quality
• Pairwise Sequential Randomization and Its Properties
• Measuring Motivations of Crowdworkers: The Multidimensional Crowdworker Motivation Scale
• LabelAId: Just-in-time AI Interventions for Improving Human Labeling Quality and Domain Knowledge in Crowdsourcing Systems
• The Online Laboratory: Conducting Experiments in a Real Labor Market
• Toward the Optimized Crowdsourcing Strategy for OCR Post-Correction
• Eliciting Worker Preference for Task Completion
• The Importance of Worker Reputation Information in Microtask-Based Crowd Work Systems
• Matching on-the-fly in Sequential Experiments for Higher Power and Efficiency
• From Zero to The Hero: A Collaborative Market Aware Recommendation System for Crowd Workers