An Active Learning Approach for Jointly Estimating Worker Performance and Annotation Reliability with Crowdsourced Data

Liyue Zhao, Yu Zhang, Gita Sukthankar

Crowdsourcing platforms offer a practical solution to the problem of affordably annotating large datasets for training supervised classifiers. Unfortunately, poor worker performanc…