MiSC: Mixed Strategies Crowdsourcing

Abstract

Popular crowdsourcing techniques mostly focus on evaluating workers' labeling quality before adjusting their weights during label aggregation. Recently, another cohort of models regard crowdsourced annotations as incomplete tensors and recover unfilled labels by tensor completion. However, mixed strategies of the two methodologies have never been comprehensively investigated, leaving them as rather independent approaches. In this work, we propose $\text{MiSC}$ ($\text{Mi}$xed $\text{S}$trategies $\text{C}$rowdsourcing), a versatile framework integrating arbitrary conventional crowdsourcing and tensor completion techniques. In particular, we propose a novel iterative Tucker label aggregation algorithm that outperforms state-of-the-art methods in extensive experiments.

arXiv comment8 pages, accepted to IJCAI 2019

References in corpus (4)

• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis
• Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination
• Incoherence-Optimal Matrix Completion