129 citations · 242 across the 18 of their papers we have counts for
5 papers · 1 filter
On the Value of ML Models
Fabio Casati, Pierre-André Noël, Jie Yang
We argue that, when establishing and benchmarking Machine Learning (ML) models, the research community should favour evaluation metrics that better capture the value delivered by t…
The Science of Rejection: A Research Area for Human Computation
Burcu Sayin, Jie Yang, Andrea Passerini +1
We motivate why the science of learning to reject model predictions is central to ML, and why human computation has a lead role in this effort.
Crowdsourcing Diverse Paraphrases for Training Task-oriented Bots
Jorge Ramírez, Auday Berro, Marcos Baez +2
A prominent approach to build datasets for training task-oriented bots is crowd-based paraphrasing. Current approaches, however, assume the crowd would naturally provide diverse pa…
On the state of reporting in crowdsourcing experiments and a checklist to aid current practices
Jorge Ramírez, Burcu Sayin, Marcos Baez +4
Crowdsourcing is being increasingly adopted as a platform to run studies with human subjects. Running a crowdsourcing experiment involves several choices and strategies to successf…
Active Hybrid Classification
Evgeny Krivosheev, Fabio Casati, Alessandro Bozzon
Hybrid crowd-machine classifiers can achieve superior performance by combining the cost-effectiveness of automatic classification with the accuracy of human judgment. This paper sh…