Evaluating Crowdsourcing Participants in the Absence of Ground-Truth
arXiv:1605.09432
Abstract
Given a supervised/semi-supervised learning scenario where multiple annotators are available, we consider the problem of identification of adversarial or unreliable annotators.
4 pages, 5 figures, Workshop on Human Computation for Science and Computational Sustainability, NIPS 2012, Lake Tahoe, NV. 7 Dec 2012