65 citations · 292 across the 21 of their papers we have counts for
3 papers · 1 filter
IMDB-WIKI-SbS: An Evaluation Dataset for Crowdsourced Pairwise Comparisons
Nikita Pavlichenko, Dmitry Ustalov
Today, comprehensive evaluation of large-scale machine learning models is possible thanks to the open datasets produced using crowdsourcing, such as SQuAD, MS COCO, ImageNet, Super…
Learning from Crowds with Crowd-Kit
Dmitry Ustalov, Nikita Pavlichenko, Boris Tseitlin
This paper presents Crowd-Kit, a general-purpose computational quality control toolkit for crowdsourcing. Crowd-Kit provides efficient and convenient implementations of popular qua…
CrowdSpeech and VoxDIY: Benchmark Datasets for Crowdsourced Audio Transcription
Nikita Pavlichenko, Ivan Stelmakh, Dmitry Ustalov
Domain-specific data is the crux of the successful transfer of machine learning systems from benchmarks to real life. In simple problems such as image classification, crowdsourcing…